2024-09-13T04:44:12.051 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T04:44:12.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T04:44:12.149 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900825' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: true rbd cache policy: writeback global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 248 sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 sleep_before_teardown: 0 subset: 116/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgscyCT753n0EMx8x0i57mgV5mIV8XX1JbPmAVNfxjR7ptTkKS7srx9kCoi4zgairRohuxGBh55PC67kgpWNDQ= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFeus21/Rbf4bNTsm1MPjb2Y8OOhR6j5yojfx5WOXHtLFmbaCGCcMaW3qpugCQTilc8RzZQIBU7G7OgwjeyVTfQ= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T04:44:12.149 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa; will attempt to use it 2024-09-13T04:44:12.150 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa/tasks 2024-09-13T04:44:12.150 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T04:44:12.151 INFO:teuthology.task.internal:Checking packages... 2024-09-13T04:44:12.175 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '48a7de7e6e1701a88fd571c7e34908ac4b8ed412' 2024-09-13T04:44:12.175 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T04:44:12.175 INFO:teuthology.packaging:ref: None 2024-09-13T04:44:12.175 INFO:teuthology.packaging:tag: None 2024-09-13T04:44:12.175 INFO:teuthology.packaging:branch: reef 2024-09-13T04:44:12.175 INFO:teuthology.packaging:sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T04:44:12.176 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-09-13T04:44:12.414 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-843-g2c01458a-1jammy 2024-09-13T04:44:12.417 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T04:44:12.424 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T04:44:12.425 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T04:44:12.437 INFO:teuthology.task.internal:Saving configuration 2024-09-13T04:44:12.449 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T04:44:12.456 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T04:44:12.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 04:38:37.024857', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMCFvA4jDs9fiwq8tu0+3TVk3f5FDtrBpyaGB7i3mhSs'} 2024-09-13T04:44:12.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 04:38:37.026583', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOM5gtbKxVd7oktLCidh0oHX0fnjzSPtrWiZegX3tqim'} 2024-09-13T04:44:12.516 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T04:44:12.522 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-13T04:44:12.522 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-13T04:44:12.522 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T04:44:12.594 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f71ebd4ba30>, signals=[15]) 2024-09-13T04:44:12.594 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T04:44:12.600 INFO:teuthology.task.internal:Opening connections... 2024-09-13T04:44:12.601 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T04:44:12.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:44:12.693 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T04:44:12.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:44:12.790 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T04:44:12.797 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-09-13T04:44:12.802 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-09-13T04:44:12.802 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="Ubuntu" 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="22.04" 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_CODENAME=jammy 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:ID=ubuntu 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE=debian 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T04:44:12.852 INFO:teuthology.orchestra.run.smithi001.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T04:44:12.853 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T04:44:12.853 INFO:teuthology.orchestra.run.smithi001.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T04:44:12.853 INFO:teuthology.orchestra.run.smithi001.stdout:UBUNTU_CODENAME=jammy 2024-09-13T04:44:12.853 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-09-13T04:44:12.885 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-09-13T04:44:12.890 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-09-13T04:44:12.890 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-09-13T04:44:12.936 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Ubuntu" 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="22.04" 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_CODENAME=jammy 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:ID=ubuntu 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE=debian 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T04:44:12.937 INFO:teuthology.orchestra.run.smithi144.stdout:UBUNTU_CODENAME=jammy 2024-09-13T04:44:12.938 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-09-13T04:44:12.968 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T04:44:12.977 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T04:44:12.983 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T04:44:12.984 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-09-13T04:44:12.986 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-09-13T04:44:12.990 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T04:44:12.997 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T04:44:12.997 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-09-13T04:44:13.033 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-09-13T04:44:13.046 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T04:44:13.115 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T04:44:13.129 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T04:44:13.129 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T04:44:13.129 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T04:44:13.130 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T04:44:13.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:44:13.135 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-13T04:44:13.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:44:13.136 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-13T04:44:13.180 INFO:teuthology.orchestra.run.smithi001.stdout:5.15.0-56-generic 2024-09-13T04:44:13.180 INFO:teuthology.task.kernel:Running kernel on smithi001: 5.15.0-56-generic 2024-09-13T04:44:13.180 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get clean 2024-09-13T04:44:13.185 INFO:teuthology.orchestra.run.smithi144.stdout:5.15.0-56-generic 2024-09-13T04:44:13.186 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.15.0-56-generic 2024-09-13T04:44:13.186 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2024-09-13T04:44:13.301 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get update 2024-09-13T04:44:13.332 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-09-13T04:44:13.449 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T04:44:13.449 INFO:teuthology.orchestra.run.smithi001.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T04:44:13.449 INFO:teuthology.orchestra.run.smithi001.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T04:44:13.481 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T04:44:13.490 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T04:44:13.515 INFO:teuthology.orchestra.run.smithi001.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T04:44:13.559 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T04:44:13.584 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T04:44:13.693 INFO:teuthology.orchestra.run.smithi001.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,805 kB] 2024-09-13T04:44:13.725 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T04:44:13.779 INFO:teuthology.orchestra.run.smithi001.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,022 kB] 2024-09-13T04:44:13.796 INFO:teuthology.orchestra.run.smithi001.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T04:44:13.798 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,022 kB] 2024-09-13T04:44:13.801 INFO:teuthology.orchestra.run.smithi001.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T04:44:13.804 INFO:teuthology.orchestra.run.smithi001.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T04:44:13.805 INFO:teuthology.orchestra.run.smithi001.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T04:44:13.825 INFO:teuthology.orchestra.run.smithi001.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T04:44:13.826 INFO:teuthology.orchestra.run.smithi001.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T04:44:13.832 INFO:teuthology.orchestra.run.smithi001.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T04:44:13.832 INFO:teuthology.orchestra.run.smithi001.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T04:44:13.835 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T04:44:13.835 INFO:teuthology.orchestra.run.smithi001.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T04:44:13.839 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T04:44:13.839 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T04:44:13.842 INFO:teuthology.orchestra.run.smithi001.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T04:44:13.844 INFO:teuthology.orchestra.run.smithi001.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T04:44:13.844 INFO:teuthology.orchestra.run.smithi001.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T04:44:13.844 INFO:teuthology.orchestra.run.smithi001.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T04:44:13.855 INFO:teuthology.orchestra.run.smithi001.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T04:44:13.867 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T04:44:13.868 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T04:44:13.873 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T04:44:13.873 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T04:44:13.879 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T04:44:13.899 INFO:teuthology.orchestra.run.smithi001.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T04:44:13.900 INFO:teuthology.orchestra.run.smithi001.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T04:44:13.910 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T04:44:13.911 INFO:teuthology.orchestra.run.smithi001.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T04:44:13.913 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T04:44:13.913 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T04:44:13.913 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T04:44:13.913 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T04:44:13.913 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T04:44:13.913 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T04:44:13.914 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T04:44:13.914 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T04:44:13.918 INFO:teuthology.orchestra.run.smithi001.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T04:44:13.919 INFO:teuthology.orchestra.run.smithi001.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T04:44:13.929 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T04:44:13.946 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T04:44:13.946 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T04:44:13.947 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T04:44:13.947 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T04:44:13.959 INFO:teuthology.orchestra.run.smithi001.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T04:44:13.960 INFO:teuthology.orchestra.run.smithi001.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T04:44:13.965 INFO:teuthology.orchestra.run.smithi001.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T04:44:13.965 INFO:teuthology.orchestra.run.smithi001.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T04:44:13.976 INFO:teuthology.orchestra.run.smithi001.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T04:44:13.982 INFO:teuthology.orchestra.run.smithi001.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T04:44:13.985 INFO:teuthology.orchestra.run.smithi001.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T04:44:13.985 INFO:teuthology.orchestra.run.smithi001.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T04:44:13.985 INFO:teuthology.orchestra.run.smithi001.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T04:44:13.986 INFO:teuthology.orchestra.run.smithi001.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T04:44:13.996 INFO:teuthology.orchestra.run.smithi001.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T04:44:13.996 INFO:teuthology.orchestra.run.smithi001.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T04:44:13.997 INFO:teuthology.orchestra.run.smithi001.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T04:44:13.997 INFO:teuthology.orchestra.run.smithi001.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T04:44:14.012 INFO:teuthology.orchestra.run.smithi001.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T04:44:14.013 INFO:teuthology.orchestra.run.smithi001.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T04:44:14.013 INFO:teuthology.orchestra.run.smithi001.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T04:44:14.013 INFO:teuthology.orchestra.run.smithi001.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T04:44:14.014 INFO:teuthology.orchestra.run.smithi001.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T04:44:14.017 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T04:44:14.301 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,805 kB] 2024-09-13T04:44:14.476 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T04:44:14.487 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T04:44:14.487 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T04:44:14.489 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T04:44:14.574 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T04:44:14.582 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T04:44:14.582 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T04:44:14.604 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T04:44:14.641 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T04:44:14.646 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T04:44:14.652 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T04:44:14.655 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T04:44:14.658 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T04:44:14.722 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T04:44:20.114 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 16.0 MB in 2s (7,398 kB/s) 2024-09-13T04:44:20.489 INFO:teuthology.orchestra.run.smithi001.stdout:Fetched 16.0 MB in 2s (7,145 kB/s) 2024-09-13T04:44:21.505 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-13T04:44:21.523 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T04:44:21.577 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-13T04:44:21.772 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-09-13T04:44:21.773 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-09-13T04:44:21.839 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-13T04:44:21.858 DEBUG:teuthology.orchestra.run.smithi001:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T04:44:21.913 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-13T04:44:21.943 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T04:44:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T04:44:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T04:44:21.943 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T04:44:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T04:44:21.944 INFO:teuthology.orchestra.run.smithi144.stdout: python2.7-minimal 2024-09-13T04:44:21.944 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T04:44:21.945 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-09-13T04:44:21.945 INFO:teuthology.orchestra.run.smithi144.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T04:44:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T04:44:21.946 INFO:teuthology.orchestra.run.smithi144.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T04:44:21.947 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-09-13T04:44:21.947 INFO:teuthology.orchestra.run.smithi144.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T04:44:21.948 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2024-09-13T04:44:21.948 INFO:teuthology.orchestra.run.smithi144.stdout: thermald 2024-09-13T04:44:21.997 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-09-13T04:44:21.997 INFO:teuthology.orchestra.run.smithi144.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T04:44:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T04:44:21.998 INFO:teuthology.orchestra.run.smithi144.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T04:44:21.999 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages will be upgraded: 2024-09-13T04:44:21.999 INFO:teuthology.orchestra.run.smithi144.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T04:44:22.110 INFO:teuthology.orchestra.run.smithi001.stdout:Building dependency tree... 2024-09-13T04:44:22.110 INFO:teuthology.orchestra.run.smithi001.stdout:Reading state information... 2024-09-13T04:44:22.178 INFO:teuthology.orchestra.run.smithi144.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T04:44:22.179 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 113 MB of archives. 2024-09-13T04:44:22.179 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T04:44:22.179 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T04:44:22.281 INFO:teuthology.orchestra.run.smithi001.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T04:44:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T04:44:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T04:44:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T04:44:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T04:44:22.282 INFO:teuthology.orchestra.run.smithi001.stdout: python2.7-minimal 2024-09-13T04:44:22.283 INFO:teuthology.orchestra.run.smithi001.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T04:44:22.284 INFO:teuthology.orchestra.run.smithi001.stdout:The following additional packages will be installed: 2024-09-13T04:44:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T04:44:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T04:44:22.284 INFO:teuthology.orchestra.run.smithi001.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T04:44:22.285 INFO:teuthology.orchestra.run.smithi001.stdout:Suggested packages: 2024-09-13T04:44:22.285 INFO:teuthology.orchestra.run.smithi001.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T04:44:22.285 INFO:teuthology.orchestra.run.smithi001.stdout:Recommended packages: 2024-09-13T04:44:22.286 INFO:teuthology.orchestra.run.smithi001.stdout: thermald 2024-09-13T04:44:22.334 INFO:teuthology.orchestra.run.smithi001.stdout:The following NEW packages will be installed: 2024-09-13T04:44:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T04:44:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T04:44:22.334 INFO:teuthology.orchestra.run.smithi001.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T04:44:22.336 INFO:teuthology.orchestra.run.smithi001.stdout:The following packages will be upgraded: 2024-09-13T04:44:22.336 INFO:teuthology.orchestra.run.smithi001.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T04:44:22.518 INFO:teuthology.orchestra.run.smithi001.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T04:44:22.518 INFO:teuthology.orchestra.run.smithi001.stdout:Need to get 113 MB of archives. 2024-09-13T04:44:22.518 INFO:teuthology.orchestra.run.smithi001.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T04:44:22.518 INFO:teuthology.orchestra.run.smithi001.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T04:44:23.564 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T04:44:23.939 INFO:teuthology.orchestra.run.smithi001.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T04:44:23.967 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T04:44:24.296 INFO:teuthology.orchestra.run.smithi001.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T04:44:26.080 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T04:44:26.081 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T04:44:26.157 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T04:44:26.494 INFO:teuthology.orchestra.run.smithi001.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T04:44:26.495 INFO:teuthology.orchestra.run.smithi001.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T04:44:26.495 INFO:teuthology.orchestra.run.smithi001.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T04:44:26.665 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T04:44:26.762 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T04:44:26.897 INFO:teuthology.orchestra.run.smithi001.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T04:44:26.941 INFO:teuthology.orchestra.run.smithi001.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T04:44:27.155 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 113 MB in 5s (23.8 MB/s) 2024-09-13T04:44:27.327 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T04:44:27.361 INFO:teuthology.orchestra.run.smithi001.stdout:Fetched 113 MB in 5s (24.6 MB/s) 2024-09-13T04:44:27.523 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T04:44:29.370 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T04:44:29.375 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T04:44:29.466 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:29.493 INFO:teuthology.orchestra.run.smithi001.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T04:44:29.498 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T04:44:29.561 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:31.662 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T04:44:31.684 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T04:44:31.763 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:32.084 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T04:44:32.105 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T04:44:32.185 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:32.657 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T04:44:32.670 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T04:44:32.678 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T04:44:32.691 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T04:44:32.715 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:32.729 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:39.009 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T04:44:39.125 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T04:44:39.433 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T04:44:39.553 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T04:44:39.589 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T04:44:39.724 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T04:44:39.831 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T04:44:39.854 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T04:44:39.898 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T04:44:40.074 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T04:44:40.203 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T04:44:40.498 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T04:44:40.520 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T04:44:40.556 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T04:44:45.613 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T04:44:45.651 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T04:44:45.679 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:46.282 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T04:44:46.319 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T04:44:46.377 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:48.331 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T04:44:48.457 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T04:44:48.769 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T04:44:49.862 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:49.988 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T04:44:50.091 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:50.657 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T04:44:50.778 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T04:44:51.175 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T04:44:51.300 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:51.416 INFO:teuthology.orchestra.run.smithi001.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T04:44:51.416 INFO:teuthology.orchestra.run.smithi001.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T04:44:51.416 INFO:teuthology.orchestra.run.smithi001.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T04:44:51.416 INFO:teuthology.orchestra.run.smithi001.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T04:44:51.426 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T04:44:51.553 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:51.585 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:52.859 INFO:teuthology.orchestra.run.smithi144.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T04:44:52.860 INFO:teuthology.orchestra.run.smithi144.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T04:44:52.860 INFO:teuthology.orchestra.run.smithi144.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T04:44:52.860 INFO:teuthology.orchestra.run.smithi144.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T04:44:53.016 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:53.028 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:54.399 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:54.446 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T04:44:54.572 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T04:44:54.706 INFO:teuthology.orchestra.run.smithi001.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:54.774 INFO:teuthology.orchestra.run.smithi001.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T04:44:54.774 INFO:teuthology.orchestra.run.smithi001.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T04:44:56.142 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T04:44:56.268 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T04:44:56.428 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T04:44:56.494 INFO:teuthology.orchestra.run.smithi144.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T04:44:56.494 INFO:teuthology.orchestra.run.smithi144.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T04:45:09.154 INFO:teuthology.orchestra.run.smithi001.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T04:45:09.154 INFO:teuthology.orchestra.run.smithi001.stdout:Sourcing file `/etc/default/grub' 2024-09-13T04:45:09.174 INFO:teuthology.orchestra.run.smithi001.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T04:45:09.192 INFO:teuthology.orchestra.run.smithi001.stdout:Generating grub configuration file ... 2024-09-13T04:45:09.449 INFO:teuthology.orchestra.run.smithi001.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T04:45:09.464 INFO:teuthology.orchestra.run.smithi001.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T04:45:09.751 INFO:teuthology.orchestra.run.smithi001.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T04:45:09.757 INFO:teuthology.orchestra.run.smithi001.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T04:45:09.838 INFO:teuthology.orchestra.run.smithi001.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T04:45:09.843 INFO:teuthology.orchestra.run.smithi001.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T04:45:09.998 INFO:teuthology.orchestra.run.smithi001.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T04:45:09.999 INFO:teuthology.orchestra.run.smithi001.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T04:45:09.999 INFO:teuthology.orchestra.run.smithi001.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T04:45:10.040 INFO:teuthology.orchestra.run.smithi001.stdout:done 2024-09-13T04:45:10.522 DEBUG:teuthology.orchestra.run.smithi001:> dpkg -s linux-image-generic 2024-09-13T04:45:10.539 INFO:teuthology.orchestra.run.smithi001.stdout:Package: linux-image-generic 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Status: install ok installed 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Priority: optional 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Section: kernel 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Installed-Size: 21 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: amd64 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Source: linux-meta 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Version: 5.15.0.121.121 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T04:45:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:Recommends: thermald 2024-09-13T04:45:10.541 INFO:teuthology.orchestra.run.smithi001.stdout:Description: Generic Linux kernel image 2024-09-13T04:45:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: This package will always depend on the latest generic kernel image 2024-09-13T04:45:10.541 INFO:teuthology.orchestra.run.smithi001.stdout: available. 2024-09-13T04:45:10.542 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T04:45:10.542 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T04:45:10.542 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T04:45:10.542 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T04:45:10.542 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T04:45:10.543 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get clean 2024-09-13T04:45:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T04:45:10.649 INFO:teuthology.orchestra.run.smithi144.stdout:Sourcing file `/etc/default/grub' 2024-09-13T04:45:10.675 INFO:teuthology.orchestra.run.smithi144.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T04:45:10.692 INFO:teuthology.orchestra.run.smithi144.stdout:Generating grub configuration file ... 2024-09-13T04:45:10.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get update 2024-09-13T04:45:10.938 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T04:45:10.957 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T04:45:11.063 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T04:45:11.063 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T04:45:11.136 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T04:45:11.217 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T04:45:11.263 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T04:45:11.269 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T04:45:11.339 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T04:45:11.344 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T04:45:11.503 INFO:teuthology.orchestra.run.smithi144.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T04:45:11.503 INFO:teuthology.orchestra.run.smithi144.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T04:45:11.504 INFO:teuthology.orchestra.run.smithi144.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T04:45:11.538 INFO:teuthology.orchestra.run.smithi144.stdout:done 2024-09-13T04:45:12.015 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2024-09-13T04:45:12.040 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2024-09-13T04:45:12.040 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2024-09-13T04:45:12.040 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2024-09-13T04:45:12.040 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2024-09-13T04:45:12.040 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 21 2024-09-13T04:45:12.040 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T04:45:12.040 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2024-09-13T04:45:12.040 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2024-09-13T04:45:12.041 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 5.15.0.121.121 2024-09-13T04:45:12.041 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T04:45:12.041 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T04:45:12.041 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2024-09-13T04:45:12.041 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2024-09-13T04:45:12.041 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2024-09-13T04:45:12.041 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2024-09-13T04:45:12.041 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T04:45:12.041 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T04:45:12.041 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T04:45:12.041 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T04:45:12.041 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T04:45:12.042 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2024-09-13T04:45:12.259 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-09-13T04:45:12.525 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T04:45:12.525 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T04:45:12.529 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-13T04:45:12.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T04:45:12.587 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T04:45:12.615 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-13T04:45:12.669 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T04:45:12.822 INFO:teuthology.orchestra.run.smithi001.stdout:Building dependency tree... 2024-09-13T04:45:12.822 INFO:teuthology.orchestra.run.smithi001.stdout:Reading state information... 2024-09-13T04:45:13.016 INFO:teuthology.orchestra.run.smithi001.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T04:45:13.016 INFO:teuthology.orchestra.run.smithi001.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T04:45:13.016 INFO:teuthology.orchestra.run.smithi001.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T04:45:13.016 INFO:teuthology.orchestra.run.smithi001.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T04:45:13.016 INFO:teuthology.orchestra.run.smithi001.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T04:45:13.017 INFO:teuthology.orchestra.run.smithi001.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T04:45:13.017 INFO:teuthology.orchestra.run.smithi001.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T04:45:13.017 INFO:teuthology.orchestra.run.smithi001.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T04:45:13.062 INFO:teuthology.orchestra.run.smithi001.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T04:45:13.064 DEBUG:teuthology.orchestra.run.smithi001:> dpkg -s linux-image-generic 2024-09-13T04:45:13.080 INFO:teuthology.orchestra.run.smithi001.stdout:Package: linux-image-generic 2024-09-13T04:45:13.080 INFO:teuthology.orchestra.run.smithi001.stdout:Status: install ok installed 2024-09-13T04:45:13.080 INFO:teuthology.orchestra.run.smithi001.stdout:Priority: optional 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Section: kernel 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Installed-Size: 21 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: amd64 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Source: linux-meta 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Version: 5.15.0.121.121 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Recommends: thermald 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:Description: Generic Linux kernel image 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout: This package will always depend on the latest generic kernel image 2024-09-13T04:45:13.081 INFO:teuthology.orchestra.run.smithi001.stdout: available. 2024-09-13T04:45:13.082 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-13T04:45:13.127 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.mB8HSv9ckp 2024-09-13T04:45:13.127 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub/grub.cfg /tmp/tmp.mB8HSv9ckp 2024-09-13T04:45:13.184 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.mB8HSv9ckp 2024-09-13T04:45:13.298 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.mB8HSv9ckp is 10KB 2024-09-13T04:45:13.312 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.mB8HSv9ckp 2024-09-13T04:45:13.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T04:45:13.371 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:45:13.371 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T04:45:13.371 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T04:45:13.440 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T04:45:13.440 DEBUG:teuthology.orchestra.run.smithi001:> sudo update-grub 2024-09-13T04:45:13.980 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-13T04:45:13.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T04:45:14.050 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-13T04:45:14.237 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-09-13T04:45:14.238 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-09-13T04:45:14.406 INFO:teuthology.orchestra.run.smithi144.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T04:45:14.406 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T04:45:14.406 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T04:45:14.406 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T04:45:14.406 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T04:45:14.407 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T04:45:14.407 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T04:45:14.407 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T04:45:14.448 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T04:45:14.450 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2024-09-13T04:45:14.507 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2024-09-13T04:45:14.507 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2024-09-13T04:45:14.507 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2024-09-13T04:45:14.507 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2024-09-13T04:45:14.507 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 21 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 5.15.0.121.121 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2024-09-13T04:45:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2024-09-13T04:45:14.509 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-09-13T04:45:14.552 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.gqtsvm2aP7 2024-09-13T04:45:14.553 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub/grub.cfg /tmp/tmp.gqtsvm2aP7 2024-09-13T04:45:14.608 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.gqtsvm2aP7 2024-09-13T04:45:14.633 INFO:teuthology.orchestra.run.smithi001.stderr:Sourcing file `/etc/default/grub' 2024-09-13T04:45:14.634 INFO:teuthology.orchestra.run.smithi001.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T04:45:14.636 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-09-13T04:45:14.731 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.gqtsvm2aP7 is 10KB 2024-09-13T04:45:14.744 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.gqtsvm2aP7 2024-09-13T04:45:14.750 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T04:45:14.804 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:45:14.804 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T04:45:14.804 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T04:45:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T04:45:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T04:45:14.871 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T04:45:14.871 DEBUG:teuthology.orchestra.run.smithi144:> sudo update-grub 2024-09-13T04:45:15.043 INFO:teuthology.orchestra.run.smithi001.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T04:45:15.048 INFO:teuthology.orchestra.run.smithi001.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T04:45:15.090 INFO:teuthology.orchestra.run.smithi001.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T04:45:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T04:45:15.229 INFO:teuthology.orchestra.run.smithi001.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T04:45:15.229 INFO:teuthology.orchestra.run.smithi001.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T04:45:15.229 INFO:teuthology.orchestra.run.smithi001.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T04:45:15.241 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-09-13T04:45:15.253 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-09-13T04:45:16.113 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub' 2024-09-13T04:45:16.114 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T04:45:16.116 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-09-13T04:45:16.283 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T04:45:16.289 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T04:45:16.547 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T04:45:16.553 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T04:45:16.593 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T04:45:16.598 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T04:45:16.726 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T04:45:16.726 INFO:teuthology.orchestra.run.smithi144.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T04:45:16.726 INFO:teuthology.orchestra.run.smithi144.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T04:45:16.736 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-09-13T04:45:16.738 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-09-13T04:45:45.280 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T04:45:45.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-13T04:45:45.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:45:46.743 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T04:45:46.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-13T04:45:46.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:46:21.249 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-09-13T04:46:30.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-13T04:46:30.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:46:45.305 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T04:46:48.637 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-09-13T04:46:54.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-13T04:46:54.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:46:57.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-09-13T04:47:00.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-13T04:47:00.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:47:01.142 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 485, in connect self._auth( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 818, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 794, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/transport.py", line 1664, in auth_publickey raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2024-09-13T04:47:05.239 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-13T04:47:06.191 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-13T04:47:06.191 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-13T04:47:06.192 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-13T04:47:06.238 INFO:teuthology.orchestra.run.smithi144.stdout:5.15.0-121-generic 2024-09-13T04:47:06.239 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T04:47:06.239 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T04:47:06.239 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T04:47:07.240 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T04:47:07.241 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T04:47:07.322 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-09-13T04:47:07.344 DEBUG:teuthology.parallel:result is None 2024-09-13T04:47:09.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-13T04:47:09.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:47:09.557 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-13T04:47:10.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-13T04:47:10.156 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-13T04:47:10.157 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-13T04:47:10.207 INFO:teuthology.orchestra.run.smithi001.stdout:5.15.0-121-generic 2024-09-13T04:47:10.208 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T04:47:10.208 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T04:47:10.208 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T04:47:11.209 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T04:47:11.209 DEBUG:teuthology.orchestra.run.smithi001:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T04:47:11.298 INFO:teuthology.orchestra.run.smithi001.stdout:ttyS1 2024-09-13T04:47:11.329 DEBUG:teuthology.parallel:result is None 2024-09-13T04:47:11.329 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T04:47:11.336 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T04:47:11.336 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T04:47:11.338 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T04:47:11.344 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T04:47:11.356 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T04:47:11.364 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T04:47:11.364 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T04:47:11.384 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T04:47:11.402 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T04:47:11.408 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T04:47:11.408 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T04:47:11.436 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T04:47:11.450 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T04:47:11.459 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T04:47:11.461 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T04:47:11.468 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T04:47:11.470 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T04:47:11.477 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T04:47:11.477 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T04:47:11.504 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T04:47:11.531 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T04:47:11.540 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T04:47:11.540 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T04:47:11.560 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T04:47:11.578 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T04:47:11.612 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T04:47:11.660 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:47:11.660 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T04:47:11.717 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T04:47:11.723 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T04:47:11.771 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:47:11.771 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T04:47:11.842 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-09-13T04:47:11.845 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-09-13T04:47:11.922 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T04:47:11.929 INFO:teuthology.task.internal:Starting timer... 2024-09-13T04:47:11.929 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T04:47:11.939 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T04:47:11.947 DEBUG:teuthology.task.selinux:Excluding smithi001: OS 'ubuntu' does not support SELinux 2024-09-13T04:47:11.947 DEBUG:teuthology.task.selinux:Excluding smithi144: OS 'ubuntu' does not support SELinux 2024-09-13T04:47:11.947 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T04:47:11.948 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T04:47:11.948 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T04:47:11.948 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T04:47:11.955 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-13T04:47:11.958 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-13T04:47:12.088 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T04:47:12.099 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T04:47:12.100 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T04:54:47.545 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-09-13T04:54:47.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-13T04:54:47.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:54:47.645 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-13T04:54:47.720 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-13T04:54:47.721 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-13T04:54:47.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:54:47.810 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-13T04:54:47.889 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-13T04:54:47.889 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T04:54:47.899 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T04:54:47.899 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T04:54:47.900 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T04:54:47.902 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T04:54:47.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T04:54:47.930 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T04:54:47.930 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Command line: ntpd -gq 2024-09-13T04:54:47.930 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: ---------------------------------------------------- 2024-09-13T04:54:47.930 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T04:54:47.930 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T04:54:47.931 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: corporation. Support and training for ntp-4 are 2024-09-13T04:54:47.931 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: available at https://www.nwtime.org/support 2024-09-13T04:54:47.931 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: ---------------------------------------------------- 2024-09-13T04:54:47.931 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: proto: precision = 0.046 usec (-24) 2024-09-13T04:54:47.931 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: basedate set to 2022-02-04 2024-09-13T04:54:47.931 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: gps base set to 2022-02-06 (week 2196) 2024-09-13T04:54:47.932 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T04:54:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T04:54:47.932 INFO:teuthology.orchestra.run.smithi001.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T04:54:47.933 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T04:54:47.933 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T04:54:47.933 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T04:54:47.933 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T04:54:47.933 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Listen normally on 3 ens1f0 172.21.15.1:123 2024-09-13T04:54:47.933 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Listen normally on 4 lo [::1]:123 2024-09-13T04:54:47.934 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15e8%4]:123 2024-09-13T04:54:47.934 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:47 ntpd[18351]: Listening on routing socket on fd #22 for interface updates 2024-09-13T04:54:47.961 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T04:54:47.962 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Command line: ntpd -gq 2024-09-13T04:54:47.962 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: ---------------------------------------------------- 2024-09-13T04:54:47.962 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T04:54:47.962 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T04:54:47.962 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: corporation. Support and training for ntp-4 are 2024-09-13T04:54:47.962 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: available at https://www.nwtime.org/support 2024-09-13T04:54:47.962 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: ---------------------------------------------------- 2024-09-13T04:54:47.963 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: proto: precision = 0.044 usec (-24) 2024-09-13T04:54:47.963 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: basedate set to 2022-02-04 2024-09-13T04:54:47.963 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: gps base set to 2022-02-06 (week 2196) 2024-09-13T04:54:47.963 INFO:teuthology.orchestra.run.smithi144.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T04:54:47.964 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T04:54:47.964 INFO:teuthology.orchestra.run.smithi144.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T04:54:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T04:54:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T04:54:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T04:54:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T04:54:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Listen normally on 3 enp3s0f1 172.21.15.144:123 2024-09-13T04:54:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Listen normally on 4 lo [::1]:123 2024-09-13T04:54:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5f%5]:123 2024-09-13T04:54:47.965 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:47 ntpd[18482]: Listening on routing socket on fd #22 for interface updates 2024-09-13T04:54:48.968 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:48.968 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:48.968 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:48.968 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:48.968 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:48.969 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:48.969 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:48.969 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:48.969 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:48.969 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:48.969 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:48.970 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:48.970 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:48.970 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:48 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:49.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:49.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:49.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:49.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:49 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:50 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:50 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:50 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:50 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:50 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:50.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:50 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:50.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:50.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:50.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:50.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:50.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:50.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:50.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:50.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:50.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:50.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:50.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:50.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:50 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:51.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:51.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:51.933 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:51.933 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:51.933 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:51.933 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:51 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:51.962 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:51 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:51.962 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:51 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:51.962 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:51 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:51.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:51 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:51.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:51 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:51.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:51 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:52.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:52 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:52.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:52 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:52.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:52 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:52.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:52 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:52.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:52 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:52.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:52 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:52.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:52.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:52.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:52.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:52.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:52.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:52.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:52.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:52.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:52.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:52.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:52.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:52 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:53.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:53.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:53 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:53.962 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:53 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:53 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:53 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:53 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:53 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:53 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:54 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:54 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:54 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:54 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:54 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:54 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:54.963 INFO:teuthology.orchestra.run.smithi144.stdout:13 Sep 04:54:54 ntpd[18482]: ntpd: time slew -0.000630 s 2024-09-13T04:54:54.964 INFO:teuthology.orchestra.run.smithi144.stdout:ntpd: time slew -0.000630s 2024-09-13T04:54:54.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T04:54:54.964 INFO:teuthology.orchestra.run.smithi144.stderr:13 Sep 04:54:54 ntpd[18482]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T04:54:55.011 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T04:54:55.011 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-09-13T04:54:55.011 INFO:teuthology.orchestra.run.smithi144.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T04:54:55.011 INFO:teuthology.orchestra.run.smithi144.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T04:54:55.011 INFO:teuthology.orchestra.run.smithi144.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T04:54:55.011 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T04:54:55.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:55.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:55.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:55.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:55.931 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T04:54:55.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T04:54:55.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T04:54:55.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T04:54:55.932 INFO:teuthology.orchestra.run.smithi001.stdout:13 Sep 04:54:55 ntpd[18351]: ntpd: time slew +0.000390 s 2024-09-13T04:54:55.932 INFO:teuthology.orchestra.run.smithi001.stdout:ntpd: time slew +0.000390s 2024-09-13T04:54:55.932 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T04:54:55.933 INFO:teuthology.orchestra.run.smithi001.stderr:13 Sep 04:54:55 ntpd[18351]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T04:54:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T04:54:55.985 INFO:teuthology.orchestra.run.smithi001.stdout:============================================================================== 2024-09-13T04:54:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T04:54:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T04:54:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T04:54:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T04:54:55.986 INFO:teuthology.run_tasks:Running task install... 2024-09-13T04:54:55.995 DEBUG:teuthology.task.install:project ceph 2024-09-13T04:54:55.995 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}} 2024-09-13T04:54:55.995 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'} 2024-09-13T04:54:55.995 INFO:teuthology.task.install:Using flavor: default 2024-09-13T04:54:56.002 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T04:54:56.002 INFO:teuthology.task.install:extra packages: [] 2024-09-13T04:54:56.002 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-key list | grep Ceph 2024-09-13T04:54:56.003 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-key list | grep Ceph 2024-09-13T04:54:56.063 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T04:54:56.082 INFO:teuthology.orchestra.run.smithi001.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T04:54:56.092 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T04:54:56.092 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T04:54:56.092 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T04:54:56.093 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T04:54:56.111 INFO:teuthology.orchestra.run.smithi001.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T04:54:56.111 INFO:teuthology.orchestra.run.smithi001.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T04:54:56.112 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T04:54:56.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T04:54:56.239 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default/ 2024-09-13T04:54:56.240 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T04:54:56.263 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default/ 2024-09-13T04:54:56.264 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T04:54:56.374 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:54:56.374 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T04:54:56.388 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-09-13T04:54:56.390 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:54:56.391 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T04:54:56.406 DEBUG:teuthology.orchestra.run.smithi001:> sudo apt-get update 2024-09-13T04:54:56.554 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T04:54:56.574 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T04:54:56.676 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T04:54:56.696 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T04:54:56.760 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T04:54:56.778 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T04:54:56.839 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T04:54:56.858 INFO:teuthology.orchestra.run.smithi001.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T04:55:18.547 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T04:55:18.567 INFO:teuthology.orchestra.run.smithi001.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T04:55:18.568 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T04:55:18.589 INFO:teuthology.orchestra.run.smithi001.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T04:55:18.590 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T04:55:18.611 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-09-13T04:55:18.612 INFO:teuthology.orchestra.run.smithi001.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T04:55:18.634 INFO:teuthology.orchestra.run.smithi001.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T04:55:18.642 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T04:55:18.666 INFO:teuthology.orchestra.run.smithi001.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-09-13T04:55:18.739 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 28.8 kB in 22s (1,298 B/s) 2024-09-13T04:55:18.748 INFO:teuthology.orchestra.run.smithi001.stdout:Fetched 28.8 kB in 22s (1,298 B/s) 2024-09-13T04:55:19.746 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-13T04:55:19.764 DEBUG:teuthology.orchestra.run.smithi001:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1jammy cephadm=18.2.4-820-g48a7de7e-1jammy ceph-mds=18.2.4-820-g48a7de7e-1jammy ceph-mgr=18.2.4-820-g48a7de7e-1jammy ceph-common=18.2.4-820-g48a7de7e-1jammy ceph-fuse=18.2.4-820-g48a7de7e-1jammy ceph-test=18.2.4-820-g48a7de7e-1jammy ceph-volume=18.2.4-820-g48a7de7e-1jammy radosgw=18.2.4-820-g48a7de7e-1jammy python3-rados=18.2.4-820-g48a7de7e-1jammy python3-rgw=18.2.4-820-g48a7de7e-1jammy python3-cephfs=18.2.4-820-g48a7de7e-1jammy python3-rbd=18.2.4-820-g48a7de7e-1jammy libcephfs2=18.2.4-820-g48a7de7e-1jammy libcephfs-dev=18.2.4-820-g48a7de7e-1jammy librados2=18.2.4-820-g48a7de7e-1jammy librbd1=18.2.4-820-g48a7de7e-1jammy rbd-fuse=18.2.4-820-g48a7de7e-1jammy 2024-09-13T04:55:19.797 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-13T04:55:19.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1jammy cephadm=18.2.4-820-g48a7de7e-1jammy ceph-mds=18.2.4-820-g48a7de7e-1jammy ceph-mgr=18.2.4-820-g48a7de7e-1jammy ceph-common=18.2.4-820-g48a7de7e-1jammy ceph-fuse=18.2.4-820-g48a7de7e-1jammy ceph-test=18.2.4-820-g48a7de7e-1jammy ceph-volume=18.2.4-820-g48a7de7e-1jammy radosgw=18.2.4-820-g48a7de7e-1jammy python3-rados=18.2.4-820-g48a7de7e-1jammy python3-rgw=18.2.4-820-g48a7de7e-1jammy python3-cephfs=18.2.4-820-g48a7de7e-1jammy python3-rbd=18.2.4-820-g48a7de7e-1jammy libcephfs2=18.2.4-820-g48a7de7e-1jammy libcephfs-dev=18.2.4-820-g48a7de7e-1jammy librados2=18.2.4-820-g48a7de7e-1jammy librbd1=18.2.4-820-g48a7de7e-1jammy rbd-fuse=18.2.4-820-g48a7de7e-1jammy 2024-09-13T04:55:19.828 INFO:teuthology.orchestra.run.smithi001.stdout:Reading package lists... 2024-09-13T04:55:19.877 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-09-13T04:55:20.038 INFO:teuthology.orchestra.run.smithi001.stdout:Building dependency tree... 2024-09-13T04:55:20.039 INFO:teuthology.orchestra.run.smithi001.stdout:Reading state information... 2024-09-13T04:55:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-09-13T04:55:20.087 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-09-13T04:55:20.255 INFO:teuthology.orchestra.run.smithi001.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T04:55:20.255 INFO:teuthology.orchestra.run.smithi001.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T04:55:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T04:55:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T04:55:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T04:55:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T04:55:20.256 INFO:teuthology.orchestra.run.smithi001.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T04:55:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:The following additional packages will be installed: 2024-09-13T04:55:20.257 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T04:55:20.257 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T04:55:20.258 INFO:teuthology.orchestra.run.smithi001.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T04:55:20.258 INFO:teuthology.orchestra.run.smithi001.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T04:55:20.258 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T04:55:20.258 INFO:teuthology.orchestra.run.smithi001.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T04:55:20.259 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T04:55:20.259 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T04:55:20.259 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T04:55:20.259 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T04:55:20.259 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T04:55:20.259 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T04:55:20.259 INFO:teuthology.orchestra.run.smithi001.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T04:55:20.260 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout:Suggested packages: 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout: python3-watchdog 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout:Recommended packages: 2024-09-13T04:55:20.261 INFO:teuthology.orchestra.run.smithi001.stdout: btrfs-tools 2024-09-13T04:55:20.279 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T04:55:20.279 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T04:55:20.279 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T04:55:20.279 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T04:55:20.279 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T04:55:20.280 INFO:teuthology.orchestra.run.smithi144.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T04:55:20.280 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T04:55:20.281 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-09-13T04:55:20.281 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T04:55:20.281 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T04:55:20.281 INFO:teuthology.orchestra.run.smithi144.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T04:55:20.281 INFO:teuthology.orchestra.run.smithi144.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T04:55:20.282 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T04:55:20.283 INFO:teuthology.orchestra.run.smithi144.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T04:55:20.283 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T04:55:20.283 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T04:55:20.283 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T04:55:20.283 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T04:55:20.283 INFO:teuthology.orchestra.run.smithi144.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T04:55:20.283 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T04:55:20.283 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout: python3-watchdog 2024-09-13T04:55:20.284 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2024-09-13T04:55:20.285 INFO:teuthology.orchestra.run.smithi144.stdout: btrfs-tools 2024-09-13T04:55:20.369 INFO:teuthology.orchestra.run.smithi001.stdout:The following NEW packages will be installed: 2024-09-13T04:55:20.369 INFO:teuthology.orchestra.run.smithi001.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T04:55:20.369 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T04:55:20.370 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T04:55:20.371 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-13T04:55:20.371 INFO:teuthology.orchestra.run.smithi001.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-13T04:55:20.371 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T04:55:20.371 INFO:teuthology.orchestra.run.smithi001.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T04:55:20.372 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T04:55:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T04:55:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T04:55:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T04:55:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T04:55:20.373 INFO:teuthology.orchestra.run.smithi001.stdout: socat xmlstarlet 2024-09-13T04:55:20.388 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-09-13T04:55:20.388 INFO:teuthology.orchestra.run.smithi144.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T04:55:20.388 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T04:55:20.388 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T04:55:20.388 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-13T04:55:20.388 INFO:teuthology.orchestra.run.smithi144.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-13T04:55:20.388 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T04:55:20.389 INFO:teuthology.orchestra.run.smithi144.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T04:55:20.389 INFO:teuthology.orchestra.run.smithi144.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T04:55:20.389 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T04:55:20.389 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-13T04:55:20.389 INFO:teuthology.orchestra.run.smithi144.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T04:55:20.389 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T04:55:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T04:55:20.391 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T04:55:20.391 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T04:55:20.391 INFO:teuthology.orchestra.run.smithi144.stdout: socat xmlstarlet 2024-09-13T04:55:20.515 INFO:teuthology.orchestra.run.smithi001.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T04:55:20.515 INFO:teuthology.orchestra.run.smithi001.stdout:Need to get 163 MB of archives. 2024-09-13T04:55:20.515 INFO:teuthology.orchestra.run.smithi001.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-13T04:55:20.516 INFO:teuthology.orchestra.run.smithi001.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1jammy [3,451 kB] 2024-09-13T04:55:20.532 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T04:55:20.532 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 163 MB of archives. 2024-09-13T04:55:20.532 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-13T04:55:20.533 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1jammy [3,451 kB] 2024-09-13T04:55:20.560 INFO:teuthology.orchestra.run.smithi001.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T04:55:20.577 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T04:55:20.716 INFO:teuthology.orchestra.run.smithi001.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1jammy [3,108 kB] 2024-09-13T04:55:20.734 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1jammy [3,108 kB] 2024-09-13T04:55:20.790 INFO:teuthology.orchestra.run.smithi001.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1jammy [683 kB] 2024-09-13T04:55:20.815 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1jammy [683 kB] 2024-09-13T04:55:20.827 INFO:teuthology.orchestra.run.smithi001.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1jammy [357 kB] 2024-09-13T04:55:20.834 INFO:teuthology.orchestra.run.smithi001.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1jammy [32.8 kB] 2024-09-13T04:55:20.835 INFO:teuthology.orchestra.run.smithi001.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1jammy [180 kB] 2024-09-13T04:55:20.839 INFO:teuthology.orchestra.run.smithi001.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1jammy [63.8 kB] 2024-09-13T04:55:20.841 INFO:teuthology.orchestra.run.smithi001.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1jammy [334 kB] 2024-09-13T04:55:20.842 INFO:teuthology.orchestra.run.smithi001.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T04:55:20.848 INFO:teuthology.orchestra.run.smithi001.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1jammy [5,934 kB] 2024-09-13T04:55:20.856 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1jammy [357 kB] 2024-09-13T04:55:20.859 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T04:55:20.863 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1jammy [32.8 kB] 2024-09-13T04:55:20.864 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1jammy [180 kB] 2024-09-13T04:55:20.868 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1jammy [63.8 kB] 2024-09-13T04:55:20.870 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1jammy [334 kB] 2024-09-13T04:55:20.877 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1jammy [5,934 kB] 2024-09-13T04:55:21.016 INFO:teuthology.orchestra.run.smithi001.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T04:55:21.030 INFO:teuthology.orchestra.run.smithi001.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T04:55:21.034 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T04:55:21.039 INFO:teuthology.orchestra.run.smithi001.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1jammy [112 kB] 2024-09-13T04:55:21.042 INFO:teuthology.orchestra.run.smithi001.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1jammy [445 kB] 2024-09-13T04:55:21.048 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T04:55:21.049 INFO:teuthology.orchestra.run.smithi001.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1jammy [22.0 MB] 2024-09-13T04:55:21.073 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1jammy [112 kB] 2024-09-13T04:55:21.075 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1jammy [445 kB] 2024-09-13T04:55:21.083 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1jammy [22.0 MB] 2024-09-13T04:55:21.104 INFO:teuthology.orchestra.run.smithi001.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T04:55:21.122 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T04:55:21.300 INFO:teuthology.orchestra.run.smithi001.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T04:55:21.309 INFO:teuthology.orchestra.run.smithi001.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T04:55:21.317 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T04:55:21.327 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T04:55:21.337 INFO:teuthology.orchestra.run.smithi001.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T04:55:21.345 INFO:teuthology.orchestra.run.smithi001.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T04:55:21.346 INFO:teuthology.orchestra.run.smithi001.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T04:55:21.347 INFO:teuthology.orchestra.run.smithi001.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T04:55:21.354 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T04:55:21.362 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T04:55:21.367 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T04:55:21.369 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T04:55:21.371 INFO:teuthology.orchestra.run.smithi001.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T04:55:21.372 INFO:teuthology.orchestra.run.smithi001.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T04:55:21.373 INFO:teuthology.orchestra.run.smithi001.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T04:55:21.383 INFO:teuthology.orchestra.run.smithi001.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T04:55:21.386 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T04:55:21.387 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T04:55:21.388 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T04:55:21.400 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T04:55:21.457 INFO:teuthology.orchestra.run.smithi001.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T04:55:21.458 INFO:teuthology.orchestra.run.smithi001.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T04:55:21.458 INFO:teuthology.orchestra.run.smithi001.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T04:55:21.459 INFO:teuthology.orchestra.run.smithi001.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T04:55:21.460 INFO:teuthology.orchestra.run.smithi001.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T04:55:21.461 INFO:teuthology.orchestra.run.smithi001.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T04:55:21.466 INFO:teuthology.orchestra.run.smithi001.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T04:55:21.467 INFO:teuthology.orchestra.run.smithi001.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T04:55:21.468 INFO:teuthology.orchestra.run.smithi001.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T04:55:21.472 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T04:55:21.473 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T04:55:21.474 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T04:55:21.474 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T04:55:21.475 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T04:55:21.476 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T04:55:21.481 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T04:55:21.482 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T04:55:21.483 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T04:55:21.546 INFO:teuthology.orchestra.run.smithi001.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T04:55:21.548 INFO:teuthology.orchestra.run.smithi001.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T04:55:21.548 INFO:teuthology.orchestra.run.smithi001.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T04:55:21.549 INFO:teuthology.orchestra.run.smithi001.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T04:55:21.551 INFO:teuthology.orchestra.run.smithi001.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T04:55:21.563 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T04:55:21.565 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T04:55:21.565 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T04:55:21.566 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T04:55:21.568 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T04:55:21.570 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T04:55:21.628 INFO:teuthology.orchestra.run.smithi001.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T04:55:21.628 INFO:teuthology.orchestra.run.smithi001.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T04:55:21.639 INFO:teuthology.orchestra.run.smithi001.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T04:55:21.640 INFO:teuthology.orchestra.run.smithi001.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T04:55:21.641 INFO:teuthology.orchestra.run.smithi001.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T04:55:21.644 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T04:55:21.655 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T04:55:21.656 INFO:teuthology.orchestra.run.smithi144.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T04:55:21.657 INFO:teuthology.orchestra.run.smithi144.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T04:55:21.709 INFO:teuthology.orchestra.run.smithi001.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T04:55:21.712 INFO:teuthology.orchestra.run.smithi001.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T04:55:21.715 INFO:teuthology.orchestra.run.smithi001.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T04:55:21.717 INFO:teuthology.orchestra.run.smithi001.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T04:55:21.720 INFO:teuthology.orchestra.run.smithi001.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T04:55:21.726 INFO:teuthology.orchestra.run.smithi144.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T04:55:21.728 INFO:teuthology.orchestra.run.smithi144.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T04:55:21.732 INFO:teuthology.orchestra.run.smithi144.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T04:55:21.734 INFO:teuthology.orchestra.run.smithi144.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T04:55:21.737 INFO:teuthology.orchestra.run.smithi144.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T04:55:21.738 INFO:teuthology.orchestra.run.smithi144.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T04:55:21.791 INFO:teuthology.orchestra.run.smithi001.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T04:55:21.807 INFO:teuthology.orchestra.run.smithi144.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T04:55:21.808 INFO:teuthology.orchestra.run.smithi144.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T04:55:21.810 INFO:teuthology.orchestra.run.smithi144.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T04:55:21.815 INFO:teuthology.orchestra.run.smithi144.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T04:55:21.840 INFO:teuthology.orchestra.run.smithi001.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T04:55:21.841 INFO:teuthology.orchestra.run.smithi001.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T04:55:21.842 INFO:teuthology.orchestra.run.smithi001.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1jammy [5,005 kB] 2024-09-13T04:55:21.843 INFO:teuthology.orchestra.run.smithi001.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T04:55:21.848 INFO:teuthology.orchestra.run.smithi001.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T04:55:21.915 INFO:teuthology.orchestra.run.smithi144.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1jammy [5,005 kB] 2024-09-13T04:55:21.935 INFO:teuthology.orchestra.run.smithi001.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1jammy [241 kB] 2024-09-13T04:55:21.940 INFO:teuthology.orchestra.run.smithi001.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1jammy [130 kB] 2024-09-13T04:55:21.943 INFO:teuthology.orchestra.run.smithi001.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1jammy [1,494 kB] 2024-09-13T04:55:21.975 INFO:teuthology.orchestra.run.smithi144.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T04:55:21.976 INFO:teuthology.orchestra.run.smithi144.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T04:55:21.978 INFO:teuthology.orchestra.run.smithi144.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T04:55:21.979 INFO:teuthology.orchestra.run.smithi144.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T04:55:21.980 INFO:teuthology.orchestra.run.smithi144.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T04:55:21.981 INFO:teuthology.orchestra.run.smithi144.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T04:55:21.981 INFO:teuthology.orchestra.run.smithi144.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T04:55:21.985 INFO:teuthology.orchestra.run.smithi001.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1jammy [6,085 kB] 2024-09-13T04:55:21.986 INFO:teuthology.orchestra.run.smithi144.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T04:55:21.986 INFO:teuthology.orchestra.run.smithi144.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T04:55:21.991 INFO:teuthology.orchestra.run.smithi001.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T04:55:21.993 INFO:teuthology.orchestra.run.smithi001.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T04:55:21.996 INFO:teuthology.orchestra.run.smithi001.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T04:55:21.996 INFO:teuthology.orchestra.run.smithi001.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T04:55:21.998 INFO:teuthology.orchestra.run.smithi001.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T04:55:21.998 INFO:teuthology.orchestra.run.smithi001.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T04:55:21.999 INFO:teuthology.orchestra.run.smithi001.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T04:55:22.001 INFO:teuthology.orchestra.run.smithi144.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1jammy [241 kB] 2024-09-13T04:55:22.003 INFO:teuthology.orchestra.run.smithi001.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T04:55:22.003 INFO:teuthology.orchestra.run.smithi001.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T04:55:22.005 INFO:teuthology.orchestra.run.smithi144.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1jammy [130 kB] 2024-09-13T04:55:22.008 INFO:teuthology.orchestra.run.smithi144.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1jammy [1,494 kB] 2024-09-13T04:55:22.054 INFO:teuthology.orchestra.run.smithi144.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1jammy [6,085 kB] 2024-09-13T04:55:22.059 INFO:teuthology.orchestra.run.smithi144.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T04:55:22.077 INFO:teuthology.orchestra.run.smithi001.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T04:55:22.096 INFO:teuthology.orchestra.run.smithi144.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T04:55:22.101 INFO:teuthology.orchestra.run.smithi144.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T04:55:22.101 INFO:teuthology.orchestra.run.smithi144.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T04:55:22.117 INFO:teuthology.orchestra.run.smithi001.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T04:55:22.121 INFO:teuthology.orchestra.run.smithi001.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T04:55:22.122 INFO:teuthology.orchestra.run.smithi001.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T04:55:22.159 INFO:teuthology.orchestra.run.smithi144.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T04:55:22.159 INFO:teuthology.orchestra.run.smithi144.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T04:55:22.160 INFO:teuthology.orchestra.run.smithi144.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T04:55:22.161 INFO:teuthology.orchestra.run.smithi144.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T04:55:22.162 INFO:teuthology.orchestra.run.smithi144.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T04:55:22.163 INFO:teuthology.orchestra.run.smithi144.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T04:55:22.178 INFO:teuthology.orchestra.run.smithi001.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T04:55:22.179 INFO:teuthology.orchestra.run.smithi001.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T04:55:22.180 INFO:teuthology.orchestra.run.smithi001.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T04:55:22.181 INFO:teuthology.orchestra.run.smithi001.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T04:55:22.181 INFO:teuthology.orchestra.run.smithi001.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T04:55:22.182 INFO:teuthology.orchestra.run.smithi001.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T04:55:22.195 INFO:teuthology.orchestra.run.smithi001.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1jammy [22.4 MB] 2024-09-13T04:55:22.222 INFO:teuthology.orchestra.run.smithi144.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T04:55:22.222 INFO:teuthology.orchestra.run.smithi144.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T04:55:22.229 INFO:teuthology.orchestra.run.smithi144.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T04:55:22.240 INFO:teuthology.orchestra.run.smithi001.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T04:55:22.241 INFO:teuthology.orchestra.run.smithi001.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T04:55:22.247 INFO:teuthology.orchestra.run.smithi001.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T04:55:22.268 INFO:teuthology.orchestra.run.smithi144.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1jammy [22.4 MB] 2024-09-13T04:55:22.303 INFO:teuthology.orchestra.run.smithi144.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T04:55:22.306 INFO:teuthology.orchestra.run.smithi144.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T04:55:22.308 INFO:teuthology.orchestra.run.smithi144.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T04:55:22.316 INFO:teuthology.orchestra.run.smithi144.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T04:55:22.321 INFO:teuthology.orchestra.run.smithi001.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T04:55:22.322 INFO:teuthology.orchestra.run.smithi144.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T04:55:22.324 INFO:teuthology.orchestra.run.smithi001.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T04:55:22.326 INFO:teuthology.orchestra.run.smithi001.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T04:55:22.332 INFO:teuthology.orchestra.run.smithi144.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T04:55:22.335 INFO:teuthology.orchestra.run.smithi001.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T04:55:22.339 INFO:teuthology.orchestra.run.smithi144.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T04:55:22.342 INFO:teuthology.orchestra.run.smithi001.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T04:55:22.353 INFO:teuthology.orchestra.run.smithi001.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T04:55:22.360 INFO:teuthology.orchestra.run.smithi001.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T04:55:22.384 INFO:teuthology.orchestra.run.smithi144.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T04:55:22.388 INFO:teuthology.orchestra.run.smithi144.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T04:55:22.403 INFO:teuthology.orchestra.run.smithi001.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T04:55:22.404 INFO:teuthology.orchestra.run.smithi144.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T04:55:22.406 INFO:teuthology.orchestra.run.smithi001.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T04:55:22.422 INFO:teuthology.orchestra.run.smithi001.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T04:55:22.466 INFO:teuthology.orchestra.run.smithi144.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T04:55:22.467 INFO:teuthology.orchestra.run.smithi144.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T04:55:22.484 INFO:teuthology.orchestra.run.smithi001.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T04:55:22.486 INFO:teuthology.orchestra.run.smithi001.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T04:55:23.022 INFO:teuthology.orchestra.run.smithi001.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1jammy [14.2 kB] 2024-09-13T04:55:23.022 INFO:teuthology.orchestra.run.smithi001.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1jammy [834 kB] 2024-09-13T04:55:23.039 INFO:teuthology.orchestra.run.smithi001.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1jammy [2,159 kB] 2024-09-13T04:55:23.073 INFO:teuthology.orchestra.run.smithi144.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1jammy [14.2 kB] 2024-09-13T04:55:23.073 INFO:teuthology.orchestra.run.smithi144.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1jammy [834 kB] 2024-09-13T04:55:23.078 INFO:teuthology.orchestra.run.smithi001.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1jammy [240 kB] 2024-09-13T04:55:23.082 INFO:teuthology.orchestra.run.smithi001.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1jammy [145 kB] 2024-09-13T04:55:23.084 INFO:teuthology.orchestra.run.smithi001.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1jammy [2,233 kB] 2024-09-13T04:55:23.088 INFO:teuthology.orchestra.run.smithi144.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1jammy [2,159 kB] 2024-09-13T04:55:23.111 INFO:teuthology.orchestra.run.smithi001.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1jammy [8,625 kB] 2024-09-13T04:55:23.123 INFO:teuthology.orchestra.run.smithi144.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1jammy [240 kB] 2024-09-13T04:55:23.127 INFO:teuthology.orchestra.run.smithi144.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1jammy [145 kB] 2024-09-13T04:55:23.129 INFO:teuthology.orchestra.run.smithi144.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1jammy [2,233 kB] 2024-09-13T04:55:23.179 INFO:teuthology.orchestra.run.smithi144.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1jammy [8,625 kB] 2024-09-13T04:55:23.384 INFO:teuthology.orchestra.run.smithi001.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1jammy [14.3 kB] 2024-09-13T04:55:23.384 INFO:teuthology.orchestra.run.smithi001.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1jammy [42.8 MB] 2024-09-13T04:55:23.460 INFO:teuthology.orchestra.run.smithi144.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1jammy [14.3 kB] 2024-09-13T04:55:23.460 INFO:teuthology.orchestra.run.smithi144.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1jammy [42.8 MB] 2024-09-13T04:55:25.505 INFO:teuthology.orchestra.run.smithi001.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1jammy [122 kB] 2024-09-13T04:55:25.573 INFO:teuthology.orchestra.run.smithi001.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1jammy [38.3 kB] 2024-09-13T04:55:25.574 INFO:teuthology.orchestra.run.smithi001.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1jammy [11.7 MB] 2024-09-13T04:55:25.774 INFO:teuthology.orchestra.run.smithi144.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1jammy [122 kB] 2024-09-13T04:55:25.776 INFO:teuthology.orchestra.run.smithi144.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1jammy [38.3 kB] 2024-09-13T04:55:25.777 INFO:teuthology.orchestra.run.smithi144.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1jammy [11.7 MB] 2024-09-13T04:55:26.062 INFO:teuthology.orchestra.run.smithi001.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1jammy [93.1 kB] 2024-09-13T04:55:26.173 INFO:teuthology.orchestra.run.smithi144.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1jammy [93.1 kB] 2024-09-13T04:55:26.663 INFO:teuthology.orchestra.run.smithi001.stdout:Fetched 163 MB in 6s (28.6 MB/s) 2024-09-13T04:55:26.735 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 163 MB in 6s (28.2 MB/s) 2024-09-13T04:55:26.804 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T04:55:26.870 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T04:55:26.905 INFO:teuthology.orchestra.run.smithi001.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T04:55:26.910 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T04:55:26.963 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T04:55:26.969 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T04:55:26.994 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T04:55:27.060 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T04:55:27.330 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T04:55:27.357 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T04:55:27.372 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T04:55:27.397 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T04:55:27.398 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T04:55:27.438 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T04:55:27.742 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T04:55:27.767 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T04:55:27.769 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T04:55:27.793 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T04:55:27.808 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T04:55:27.833 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T04:55:28.095 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T04:55:28.110 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T04:55:28.144 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T04:55:28.178 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T04:55:28.204 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T04:55:28.245 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T04:55:28.598 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T04:55:28.625 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T04:55:28.673 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:55:28.674 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T04:55:28.700 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T04:55:28.741 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:55:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T04:55:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T04:55:29.135 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T04:55:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:55:29.161 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T04:55:29.202 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:55:29.487 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T04:55:29.514 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T04:55:29.546 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T04:55:29.554 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:55:29.572 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T04:55:29.613 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:55:29.849 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T04:55:29.875 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T04:55:29.907 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T04:55:29.941 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T04:55:29.967 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T04:55:29.999 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T04:55:30.268 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package librados2. 2024-09-13T04:55:30.295 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:30.335 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:30.344 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librados2. 2024-09-13T04:55:30.371 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:30.411 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:30.847 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librbd1. 2024-09-13T04:55:30.873 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../009-librbd1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:30.887 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package librbd1. 2024-09-13T04:55:30.913 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:30.914 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../009-librbd1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:30.954 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:31.358 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T04:55:31.385 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:31.407 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T04:55:31.425 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:31.434 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:31.473 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:31.686 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rados. 2024-09-13T04:55:31.712 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../011-python3-rados_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:31.744 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:31.767 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-rados. 2024-09-13T04:55:31.783 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../011-python3-rados_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:31.826 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:32.038 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T04:55:32.065 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:55:32.105 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:32.112 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T04:55:32.139 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:55:32.170 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:32.366 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T04:55:32.381 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:32.389 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T04:55:32.405 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:32.424 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:32.456 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:32.744 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T04:55:32.758 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T04:55:32.770 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:55:32.785 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:55:32.810 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:32.825 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:33.097 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T04:55:33.123 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T04:55:33.153 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T04:55:33.155 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T04:55:33.180 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T04:55:33.219 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T04:55:33.416 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T04:55:33.432 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T04:55:33.466 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T04:55:33.539 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T04:55:33.566 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T04:55:33.605 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T04:55:33.719 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T04:55:33.734 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:33.777 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:33.844 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T04:55:33.859 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:33.893 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:34.155 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T04:55:34.181 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T04:55:34.221 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T04:55:34.230 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T04:55:34.256 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T04:55:34.296 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T04:55:34.558 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librgw2. 2024-09-13T04:55:34.584 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:34.607 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package librgw2. 2024-09-13T04:55:34.624 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:34.634 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:34.674 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:35.102 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T04:55:35.117 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:35.152 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:35.202 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T04:55:35.228 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:35.268 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:35.496 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T04:55:35.523 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T04:55:35.537 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T04:55:35.563 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T04:55:35.564 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T04:55:35.604 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T04:55:35.887 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T04:55:35.913 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:35.958 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:36.049 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T04:55:36.075 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:36.115 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:36.320 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-common. 2024-09-13T04:55:36.346 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:36.377 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:36.544 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-common. 2024-09-13T04:55:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:36.610 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:37.683 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-base. 2024-09-13T04:55:37.710 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:37.759 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:37.949 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-base. 2024-09-13T04:55:37.976 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:38.025 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:38.388 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T04:55:38.414 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T04:55:38.454 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T04:55:38.612 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T04:55:38.638 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T04:55:38.678 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T04:55:38.757 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T04:55:38.783 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T04:55:38.824 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T04:55:38.981 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T04:55:39.008 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T04:55:39.047 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T04:55:39.152 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T04:55:39.178 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T04:55:39.218 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T04:55:39.342 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T04:55:39.369 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T04:55:39.409 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T04:55:39.479 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T04:55:39.495 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T04:55:39.529 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T04:55:39.645 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T04:55:39.658 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T04:55:39.695 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T04:55:39.824 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T04:55:39.851 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T04:55:39.891 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T04:55:39.990 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T04:55:40.016 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T04:55:40.056 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T04:55:40.194 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T04:55:40.220 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T04:55:40.260 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T04:55:40.367 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T04:55:40.394 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T04:55:40.434 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T04:55:40.538 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-portend. 2024-09-13T04:55:40.564 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-13T04:55:40.605 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T04:55:40.695 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-portend. 2024-09-13T04:55:40.722 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-13T04:55:40.753 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T04:55:40.883 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T04:55:40.909 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T04:55:40.949 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T04:55:41.039 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T04:55:41.066 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T04:55:41.106 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T04:55:41.261 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T04:55:41.287 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T04:55:41.327 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T04:55:41.584 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T04:55:41.611 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T04:55:41.614 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T04:55:41.629 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T04:55:41.650 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T04:55:41.672 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T04:55:41.996 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T04:55:42.009 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T04:55:42.022 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T04:55:42.035 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T04:55:42.062 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T04:55:42.075 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T04:55:42.373 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T04:55:42.378 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T04:55:42.400 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T04:55:42.405 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T04:55:42.440 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T04:55:42.445 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T04:55:42.698 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-mako. 2024-09-13T04:55:42.724 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T04:55:42.743 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T04:55:42.756 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T04:55:42.770 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T04:55:42.809 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T04:55:43.056 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-mako. 2024-09-13T04:55:43.059 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T04:55:43.070 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T04:55:43.086 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T04:55:43.104 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T04:55:43.126 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T04:55:43.407 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T04:55:43.434 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T04:55:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T04:55:43.654 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T04:55:43.680 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T04:55:43.720 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T04:55:43.776 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T04:55:43.803 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T04:55:43.843 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T04:55:44.023 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webob. 2024-09-13T04:55:44.049 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T04:55:44.098 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T04:55:44.137 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-webob. 2024-09-13T04:55:44.164 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T04:55:44.204 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T04:55:44.460 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T04:55:44.486 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T04:55:44.529 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T04:55:44.532 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T04:55:44.559 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T04:55:44.602 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T04:55:44.804 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T04:55:44.830 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T04:55:44.862 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T04:55:44.910 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T04:55:44.936 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T04:55:44.968 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T04:55:45.149 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-paste. 2024-09-13T04:55:45.175 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T04:55:45.215 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T04:55:45.271 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-paste. 2024-09-13T04:55:45.298 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T04:55:45.337 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T04:55:45.518 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T04:55:45.534 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T04:55:45.577 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T04:55:45.624 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T04:55:45.640 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T04:55:45.673 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T04:55:45.879 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T04:55:45.906 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T04:55:45.946 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T04:55:45.951 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T04:55:45.979 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T04:55:46.018 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T04:55:46.208 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T04:55:46.223 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T04:55:46.266 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T04:55:46.321 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T04:55:46.348 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T04:55:46.387 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T04:55:46.577 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T04:55:46.603 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T04:55:46.643 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T04:55:46.716 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T04:55:46.745 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T04:55:46.782 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T04:55:47.007 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T04:55:47.033 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T04:55:47.073 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T04:55:47.120 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T04:55:47.148 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T04:55:47.187 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T04:55:47.377 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-idna. 2024-09-13T04:55:47.403 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T04:55:47.435 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T04:55:47.473 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-idna. 2024-09-13T04:55:47.500 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T04:55:47.539 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T04:55:47.738 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T04:55:47.764 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T04:55:47.804 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T04:55:47.851 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T04:55:47.878 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T04:55:47.917 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T04:55:48.116 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests. 2024-09-13T04:55:48.143 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T04:55:48.183 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T04:55:48.271 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-requests. 2024-09-13T04:55:48.298 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T04:55:48.337 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T04:55:48.494 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T04:55:48.520 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T04:55:48.560 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T04:55:48.648 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T04:55:48.676 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T04:55:48.715 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T04:55:48.855 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T04:55:48.882 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:55:48.922 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:48.985 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T04:55:49.012 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:55:49.051 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:49.310 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T04:55:49.337 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:49.377 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:49.448 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T04:55:49.475 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:49.514 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T04:55:49.750 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T04:55:49.756 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T04:55:49.766 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T04:55:49.796 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T04:55:49.808 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T04:55:50.108 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T04:55:50.111 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T04:55:50.134 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T04:55:50.139 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T04:55:50.174 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T04:55:50.178 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T04:55:50.469 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T04:55:50.495 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:50.527 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:50.539 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T04:55:50.567 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:50.606 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:50.930 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T04:55:50.951 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T04:55:50.957 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:50.978 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:50.997 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:51.009 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:51.617 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T04:55:51.644 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:51.684 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:51.821 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T04:55:51.848 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:52.812 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph. 2024-09-13T04:55:52.839 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../060-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:52.870 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:52.932 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph. 2024-09-13T04:55:52.948 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../060-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:52.990 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:53.181 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T04:55:53.208 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:53.248 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:53.321 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T04:55:53.345 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:53.376 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:53.643 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T04:55:53.669 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:53.709 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:53.730 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T04:55:53.757 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:53.796 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:54.063 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephadm. 2024-09-13T04:55:54.089 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../063-cephadm_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:54.121 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:54.183 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package cephadm. 2024-09-13T04:55:54.210 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../063-cephadm_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:55:54.249 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:55.069 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T04:55:55.097 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T04:55:55.116 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T04:55:55.143 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T04:55:55.182 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T04:55:55.310 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T04:55:55.469 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T04:55:55.496 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T04:55:55.535 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T04:55:55.613 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T04:55:55.641 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T04:55:55.680 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T04:55:56.568 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-babel. 2024-09-13T04:55:56.595 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T04:55:56.634 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T04:55:56.677 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-babel. 2024-09-13T04:55:56.704 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T04:55:56.743 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T04:55:56.937 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T04:55:56.965 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T04:55:56.995 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T04:55:57.063 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T04:55:57.090 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T04:55:57.129 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T04:55:57.332 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T04:55:57.359 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:55:57.390 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:57.499 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T04:55:57.527 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:55:57.566 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:55:57.694 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T04:55:57.710 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T04:55:57.744 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T04:55:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T04:55:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T04:55:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T04:55:58.039 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-py. 2024-09-13T04:55:58.066 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-13T04:55:58.105 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T04:55:58.297 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-py. 2024-09-13T04:55:58.314 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-13T04:55:58.355 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T04:55:58.434 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T04:55:58.461 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T04:55:58.500 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T04:55:58.675 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T04:55:58.702 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T04:55:58.742 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T04:55:58.803 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-toml. 2024-09-13T04:55:58.830 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-13T04:55:58.870 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T04:55:59.003 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-toml. 2024-09-13T04:55:59.031 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-13T04:55:59.061 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T04:55:59.131 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T04:55:59.158 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T04:55:59.189 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T04:55:59.339 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T04:55:59.367 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T04:55:59.405 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T04:55:59.535 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T04:55:59.562 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T04:55:59.601 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T04:55:59.709 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T04:55:59.737 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T04:55:59.767 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T04:55:59.863 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-routes. 2024-09-13T04:55:59.890 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T04:55:59.921 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T04:56:00.046 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-routes. 2024-09-13T04:56:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T04:56:00.112 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T04:56:00.258 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T04:56:00.288 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:56:00.316 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:00.457 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T04:56:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:56:00.524 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:01.588 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T04:56:01.604 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T04:56:01.646 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T04:56:01.697 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T04:56:01.712 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T04:56:01.745 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T04:56:02.149 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T04:56:02.165 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T04:56:02.199 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T04:56:02.416 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T04:56:02.443 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T04:56:02.482 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T04:56:02.562 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T04:56:02.589 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T04:56:02.628 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T04:56:02.853 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T04:56:02.880 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T04:56:02.919 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T04:56:02.923 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T04:56:02.950 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T04:56:02.989 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T04:56:03.180 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T04:56:03.208 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T04:56:03.247 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T04:56:03.648 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T04:56:03.675 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:56:03.714 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:03.947 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T04:56:03.976 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:56:04.013 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:04.917 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T04:56:04.944 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T04:56:04.949 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T04:56:04.977 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T04:56:04.983 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T04:56:05.007 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T04:56:05.364 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T04:56:05.371 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T04:56:05.389 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T04:56:05.390 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T04:56:05.429 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T04:56:05.430 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T04:56:05.732 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T04:56:05.759 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T04:56:05.767 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T04:56:05.794 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T04:56:05.799 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T04:56:05.834 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T04:56:06.110 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T04:56:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T04:56:06.137 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-13T04:56:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-13T04:56:06.176 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T04:56:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T04:56:06.472 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T04:56:06.488 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T04:56:06.507 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T04:56:06.521 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T04:56:06.534 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T04:56:06.573 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T04:56:06.808 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T04:56:06.836 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T04:56:06.852 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T04:56:06.875 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T04:56:06.879 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T04:56:06.910 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T04:56:07.203 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T04:56:07.205 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T04:56:07.231 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T04:56:07.232 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T04:56:07.269 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T04:56:07.271 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T04:56:07.591 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T04:56:07.618 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T04:56:07.657 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T04:56:07.789 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T04:56:07.817 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T04:56:07.856 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T04:56:08.263 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T04:56:08.277 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:56:08.321 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:08.537 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T04:56:08.565 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:56:08.603 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:08.668 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T04:56:08.697 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T04:56:08.741 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T04:56:08.922 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T04:56:08.950 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T04:56:08.989 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T04:56:09.086 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T04:56:09.114 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T04:56:09.152 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T04:56:09.325 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T04:56:09.349 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T04:56:09.388 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package jq. 2024-09-13T04:56:09.391 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T04:56:09.404 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T04:56:09.438 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T04:56:09.602 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package jq. 2024-09-13T04:56:09.619 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T04:56:09.652 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T04:56:09.674 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package socat. 2024-09-13T04:56:09.690 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T04:56:09.732 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T04:56:09.905 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package socat. 2024-09-13T04:56:09.921 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T04:56:09.963 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T04:56:10.044 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T04:56:10.061 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T04:56:10.111 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T04:56:10.283 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T04:56:10.299 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T04:56:10.341 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T04:56:10.465 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-test. 2024-09-13T04:56:10.493 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../096-ceph-test_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:56:10.531 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:10.723 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-test. 2024-09-13T04:56:10.751 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../096-ceph-test_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:56:10.789 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:12.469 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T04:56:12.496 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:56:12.544 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:12.710 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T04:56:12.738 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T04:56:12.776 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:12.831 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T04:56:12.847 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:56:12.881 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:13.080 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T04:56:13.097 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:56:13.146 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:13.217 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T04:56:13.245 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T04:56:13.284 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T04:56:13.525 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T04:56:13.553 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T04:56:13.591 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T04:56:13.614 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T04:56:13.639 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T04:56:13.681 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T04:56:13.912 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T04:56:13.929 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T04:56:13.970 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T04:56:14.142 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T04:56:14.170 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T04:56:14.208 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T04:56:14.374 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T04:56:14.402 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T04:56:14.440 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T04:56:14.529 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T04:56:14.556 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T04:56:14.595 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T04:56:14.752 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T04:56:14.781 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T04:56:14.819 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T04:56:14.940 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T04:56:14.968 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T04:56:15.006 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T04:56:15.155 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T04:56:15.183 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T04:56:15.238 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T04:56:15.477 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T04:56:15.505 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T04:56:15.544 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T04:56:15.701 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T04:56:15.729 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T04:56:15.766 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T04:56:15.855 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T04:56:15.883 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T04:56:15.921 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T04:56:16.080 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T04:56:16.106 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T04:56:16.144 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T04:56:16.174 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T04:56:16.202 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T04:56:16.240 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T04:56:16.431 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T04:56:16.459 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T04:56:16.497 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T04:56:17.003 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package radosgw. 2024-09-13T04:56:17.031 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../107-radosgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:56:17.069 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:17.134 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package radosgw. 2024-09-13T04:56:17.163 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../107-radosgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:56:17.192 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:17.781 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T04:56:17.797 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:56:17.830 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:18.134 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T04:56:18.162 INFO:teuthology.orchestra.run.smithi001.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T04:56:18.190 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T04:56:18.228 INFO:teuthology.orchestra.run.smithi001.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:18.385 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T04:56:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T04:56:18.524 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T04:56:18.746 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T04:56:18.783 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T04:56:18.909 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T04:56:19.111 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T04:56:19.177 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T04:56:19.508 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T04:56:19.551 INFO:teuthology.orchestra.run.smithi144.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T04:56:19.638 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:19.916 INFO:teuthology.orchestra.run.smithi001.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T04:56:19.986 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:20.447 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user cephadm....done 2024-09-13T04:56:20.511 INFO:teuthology.orchestra.run.smithi001.stdout:Adding system user cephadm....done 2024-09-13T04:56:20.578 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T04:56:20.659 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T04:56:20.854 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T04:56:20.926 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T04:56:21.114 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T04:56:21.186 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T04:56:21.240 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T04:56:21.312 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T04:56:21.491 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T04:56:21.563 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T04:56:21.742 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T04:56:21.823 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T04:56:21.860 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T04:56:21.949 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T04:56:22.162 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T04:56:22.259 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T04:56:22.522 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T04:56:22.619 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T04:56:22.773 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T04:56:22.879 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T04:56:23.033 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T04:56:23.146 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T04:56:23.309 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T04:56:23.431 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T04:56:23.560 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T04:56:23.690 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T04:56:23.803 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:23.966 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:24.162 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T04:56:24.217 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T04:56:24.305 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T04:56:24.460 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T04:56:24.548 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T04:56:24.673 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T04:56:24.728 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T04:56:24.879 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T04:56:25.067 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T04:56:25.201 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T04:56:25.247 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T04:56:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T04:56:25.452 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T04:56:25.640 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T04:56:25.720 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T04:56:25.908 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T04:56:26.029 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T04:56:26.209 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T04:56:26.706 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T04:56:26.953 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T04:56:26.966 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T04:56:27.092 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T04:56:27.195 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T04:56:27.226 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T04:56:27.322 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T04:56:27.448 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T04:56:27.528 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T04:56:27.757 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T04:56:27.895 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T04:56:28.156 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T04:56:28.175 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T04:56:28.418 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T04:56:28.523 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T04:56:28.799 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T04:56:28.811 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T04:56:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T04:56:29.168 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T04:56:29.410 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T04:56:29.464 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T04:56:29.519 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:29.723 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T04:56:29.821 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T04:56:29.849 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:30.134 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T04:56:30.138 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T04:56:30.459 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T04:56:31.295 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:56:31.429 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T04:56:31.615 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:56:31.680 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T04:56:31.741 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T04:56:31.781 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T04:56:31.898 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T04:56:31.992 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T04:56:32.118 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T04:56:32.216 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T04:56:32.244 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T04:56:32.501 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T04:56:32.529 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T04:56:32.752 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:56:32.789 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T04:56:32.870 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T04:56:33.021 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T04:56:33.039 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:56:33.166 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T04:56:33.291 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T04:56:33.339 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T04:56:33.598 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T04:56:33.601 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T04:56:33.858 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T04:56:33.860 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T04:56:34.118 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T04:56:34.128 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T04:56:34.371 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T04:56:34.385 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T04:56:34.622 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T04:56:34.645 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T04:56:34.771 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T04:56:34.899 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T04:56:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T04:56:35.038 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T04:56:35.165 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T04:56:35.300 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T04:56:35.426 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T04:56:35.499 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T04:56:35.752 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T04:56:35.833 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T04:56:36.054 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T04:56:36.093 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T04:56:36.304 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T04:56:36.448 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T04:56:36.698 INFO:teuthology.orchestra.run.smithi001.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T04:56:36.704 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T04:56:36.830 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T04:56:36.974 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T04:56:37.075 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T04:56:37.223 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T04:56:37.460 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T04:56:37.491 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T04:56:37.753 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T04:56:37.759 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:56:37.860 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T04:56:38.012 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T04:56:38.138 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T04:56:38.353 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T04:56:38.473 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T04:56:38.637 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T04:56:38.841 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T04:56:38.922 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T04:56:39.048 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T04:56:39.108 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T04:56:39.234 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T04:56:39.332 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T04:56:39.502 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T04:56:39.600 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T04:56:39.770 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T04:56:39.910 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T04:56:40.046 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T04:56:40.286 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T04:56:40.422 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T04:56:40.554 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T04:56:40.732 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T04:56:40.873 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T04:56:40.999 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.084 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T04:56:41.125 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.210 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.251 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.336 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.360 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.470 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.613 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.681 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T04:56:41.739 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:41.942 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T04:56:41.957 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:42.084 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:42.202 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:42.212 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:42.320 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:42.410 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T04:56:42.410 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T04:56:42.420 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:42.622 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T04:56:42.622 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T04:56:42.938 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:43.065 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:43.131 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:43.190 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T04:56:43.290 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:43.433 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T04:56:44.420 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:44.521 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:44.622 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:44.740 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:44.791 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:44.866 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:45.029 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:45.154 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:45.255 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:45.390 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:45.400 INFO:teuthology.orchestra.run.smithi144.stdout:Adding group ceph....done 2024-09-13T04:56:45.751 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user ceph....done 2024-09-13T04:56:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:Adding group ceph....done 2024-09-13T04:56:45.945 INFO:teuthology.orchestra.run.smithi144.stdout:Setting system user ceph properties....done 2024-09-13T04:56:45.959 INFO:teuthology.orchestra.run.smithi144.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T04:56:46.109 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T04:56:46.284 INFO:teuthology.orchestra.run.smithi001.stdout:Adding system user ceph....done 2024-09-13T04:56:46.444 INFO:teuthology.orchestra.run.smithi001.stdout:Setting system user ceph properties....done 2024-09-13T04:56:46.452 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T04:56:46.459 INFO:teuthology.orchestra.run.smithi001.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T04:56:46.606 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T04:56:46.950 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:46.968 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T04:56:47.076 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:47.424 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:47.548 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T04:56:47.548 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T04:56:47.550 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:48.024 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T04:56:48.025 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T04:56:48.077 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:48.469 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:48.526 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T04:56:48.806 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T04:56:49.006 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:49.228 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T04:56:49.228 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T04:56:49.262 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:49.490 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T04:56:49.490 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T04:56:49.716 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:49.932 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T04:56:49.932 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T04:56:49.947 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:50.178 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T04:56:50.178 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T04:56:50.417 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:50.682 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:50.685 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T04:56:50.685 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T04:56:50.961 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T04:56:50.961 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T04:56:51.194 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:51.303 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:51.429 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:51.467 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:51.609 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:51.624 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T04:56:51.624 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T04:56:51.761 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:51.973 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T04:56:51.973 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T04:56:52.105 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:52.248 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:52.374 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:52.428 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:52.525 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:52.563 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:52.681 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:52.807 INFO:teuthology.orchestra.run.smithi001.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T04:56:52.853 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T04:56:53.259 INFO:teuthology.orchestra.run.smithi001.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T04:56:54.055 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T04:56:54.144 INFO:teuthology.orchestra.run.smithi001.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T04:56:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T04:56:54.547 DEBUG:teuthology.parallel:result is None 2024-09-13T04:56:54.603 INFO:teuthology.orchestra.run.smithi001.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T04:56:54.608 DEBUG:teuthology.parallel:result is None 2024-09-13T04:56:54.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T04:56:54.767 DEBUG:teuthology.orchestra.run.smithi001:> dpkg-query -W -f '${Version}' ceph 2024-09-13T04:56:54.787 INFO:teuthology.orchestra.run.smithi001.stdout:18.2.4-820-g48a7de7e-1jammy 2024-09-13T04:56:54.787 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T04:56:54.787 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1jammy is installed. 2024-09-13T04:56:54.789 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T04:56:54.940 DEBUG:teuthology.orchestra.run.smithi144:> dpkg-query -W -f '${Version}' ceph 2024-09-13T04:56:54.959 INFO:teuthology.orchestra.run.smithi144.stdout:18.2.4-820-g48a7de7e-1jammy 2024-09-13T04:56:54.959 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T04:56:54.960 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1jammy is installed. 2024-09-13T04:56:54.962 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T04:56:54.962 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:54.962 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T04:56:54.977 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:56:54.978 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T04:56:55.013 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T04:56:55.014 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:55.014 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T04:56:55.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T04:56:55.089 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:56:55.089 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T04:56:55.102 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T04:56:55.156 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T04:56:55.156 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:55.156 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T04:56:55.170 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T04:56:55.225 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:56:55.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T04:56:55.238 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T04:56:55.292 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T04:56:55.292 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:55.292 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T04:56:55.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T04:56:55.362 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:56:55.362 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T04:56:55.375 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T04:56:55.427 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T04:56:55.514 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T04:56:55.514 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /var/log/ceph 2024-09-13T04:56:55.517 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2024-09-13T04:56:55.530 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T04:56:55.530 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T04:56:55.571 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T04:56:55.583 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T04:56:55.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T04:56:55.627 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T04:56:55.650 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T04:56:55.650 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T04:56:55.651 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900825', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 248, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'sleep_before_teardown': 0, 'subset': '116/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880', 'targets': {'smithi001.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgscyCT753n0EMx8x0i57mgV5mIV8XX1JbPmAVNfxjR7ptTkKS7srx9kCoi4zgairRohuxGBh55PC67kgpWNDQ=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFeus21/Rbf4bNTsm1MPjb2Y8OOhR6j5yojfx5WOXHtLFmbaCGCcMaW3qpugCQTilc8RzZQIBU7G7OgwjeyVTfQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T04:56:55.651 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T04:56:55.682 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T04:56:55.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T04:56:55.734 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T04:56:55.748 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:55.748 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T04:56:55.793 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T04:56:55.794 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-09-13T04:56:55.846 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T04:56:55.846 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T04:56:55.847 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-09-13T04:56:55.847 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T04:56:55.847 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-13 04:51:15.472856778 +0000 2024-09-13T04:56:55.847 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-13 04:51:15.232858921 +0000 2024-09-13T04:56:55.847 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-13 04:51:15.232858921 +0000 2024-09-13T04:56:55.847 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2024-09-13T04:56:55.848 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T04:56:55.900 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-13T04:56:55.900 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-13T04:56:55.900 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000323375 s, 1.6 MB/s 2024-09-13T04:56:55.902 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T04:56:55.951 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-09-13T04:56:55.998 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T04:56:55.998 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T04:56:55.998 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-09-13T04:56:55.998 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T04:56:55.998 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-13 04:51:15.944852562 +0000 2024-09-13T04:56:55.998 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-13 04:51:15.708854670 +0000 2024-09-13T04:56:55.998 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-13 04:51:15.708854670 +0000 2024-09-13T04:56:55.998 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2024-09-13T04:56:55.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T04:56:56.052 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-13T04:56:56.052 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-13T04:56:56.052 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000315294 s, 1.6 MB/s 2024-09-13T04:56:56.053 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T04:56:56.103 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-09-13T04:56:56.149 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T04:56:56.150 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T04:56:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-09-13T04:56:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T04:56:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-13 04:51:16.436848166 +0000 2024-09-13T04:56:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-13 04:51:16.168850561 +0000 2024-09-13T04:56:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-13 04:51:16.168850561 +0000 2024-09-13T04:56:56.150 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2024-09-13T04:56:56.150 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T04:56:56.204 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-13T04:56:56.204 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-13T04:56:56.204 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000308141 s, 1.7 MB/s 2024-09-13T04:56:56.205 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T04:56:56.255 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-09-13T04:56:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T04:56:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T04:56:56.302 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-09-13T04:56:56.302 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T04:56:56.302 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-13 04:51:16.668846094 +0000 2024-09-13T04:56:56.302 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-13 04:51:16.664846130 +0000 2024-09-13T04:56:56.302 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-13 04:51:16.664846130 +0000 2024-09-13T04:56:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2024-09-13T04:56:56.303 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T04:56:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-13T04:56:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-13T04:56:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000280382 s, 1.8 MB/s 2024-09-13T04:56:56.357 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T04:56:56.407 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T04:56:56.407 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:56:56.407 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T04:56:56.413 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T04:56:56.413 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-09-13T04:56:56.461 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T04:56:56.461 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T04:56:56.461 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 872 Links: 1 2024-09-13T04:56:56.461 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T04:56:56.461 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-13 04:51:11.590408672 +0000 2024-09-13T04:56:56.461 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-13 04:51:11.334410952 +0000 2024-09-13T04:56:56.461 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-13 04:51:11.334410952 +0000 2024-09-13T04:56:56.462 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-13T04:56:56.462 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T04:56:56.515 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-13T04:56:56.516 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-13T04:56:56.516 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000370373 s, 1.4 MB/s 2024-09-13T04:56:56.517 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T04:56:56.566 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-09-13T04:56:56.613 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T04:56:56.613 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T04:56:56.614 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-09-13T04:56:56.614 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T04:56:56.614 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-13 04:51:12.098404146 +0000 2024-09-13T04:56:56.614 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-13 04:51:11.838406462 +0000 2024-09-13T04:56:56.614 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-13 04:51:11.838406462 +0000 2024-09-13T04:56:56.615 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-13T04:56:56.615 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T04:56:56.667 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-13T04:56:56.667 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-13T04:56:56.667 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.00043882 s, 1.2 MB/s 2024-09-13T04:56:56.668 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T04:56:56.718 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-09-13T04:56:56.764 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T04:56:56.765 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T04:56:56.765 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-09-13T04:56:56.765 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T04:56:56.765 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-13 04:51:12.630399405 +0000 2024-09-13T04:56:56.765 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-13 04:51:12.378401651 +0000 2024-09-13T04:56:56.765 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-13 04:51:12.378401651 +0000 2024-09-13T04:56:56.765 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-13T04:56:56.765 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T04:56:56.818 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-13T04:56:56.819 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-13T04:56:56.819 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000306129 s, 1.7 MB/s 2024-09-13T04:56:56.820 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T04:56:56.870 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-09-13T04:56:56.917 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T04:56:56.917 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T04:56:56.917 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-09-13T04:56:56.917 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T04:56:56.917 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-13 04:51:13.126394986 +0000 2024-09-13T04:56:56.917 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-13 04:51:12.862397338 +0000 2024-09-13T04:56:56.917 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-13 04:51:12.862397338 +0000 2024-09-13T04:56:56.917 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-13T04:56:56.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T04:56:56.970 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-13T04:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-13T04:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000279751 s, 1.8 MB/s 2024-09-13T04:56:56.972 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T04:56:57.022 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T04:56:57.022 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T04:56:57.022 INFO:tasks.ceph:Generating config... 2024-09-13T04:56:57.023 INFO:tasks.ceph:[client] rbd cache = True 2024-09-13T04:56:57.023 INFO:tasks.ceph:[client] rbd cache policy = writeback 2024-09-13T04:56:57.023 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T04:56:57.024 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T04:56:57.024 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T04:56:57.024 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T04:56:57.024 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T04:56:57.024 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T04:56:57.024 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T04:56:57.024 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T04:56:57.024 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T04:56:57.024 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-09-13T04:56:57.024 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-13T04:56:57.024 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T04:56:57.024 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T04:56:57.025 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-13T04:56:57.026 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T04:56:57.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T04:56:57.061 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T04:56:57.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T04:56:57.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T04:56:57.205 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.1'), ('mon.c', '[v2:172.21.15.1:3301,v1:172.21.15.1:6790]'), ('mon.b', '172.21.15.144')] 2024-09-13T04:56:57.205 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.144', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writeback'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-13T04:56:57.206 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:57.206 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T04:56:57.253 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.1 --addv c '[v2:172.21.15.1:3301,v1:172.21.15.1:6790]' --add b 172.21.15.144 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:56:57.341 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:56:57.341 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: generated fsid 8850da77-4384-4335-9559-af2e805c4f72 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = pacific 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 0 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 8850da77-4384-4335-9559-af2e805c4f72 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:last_changed 2024-09-13T04:56:57.342675+0000 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-13T04:56:57.342675+0000 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:min_mon_release 16 (pacific) 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:election_strategy: 1 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:2: [v2:172.21.15.1:3301/0,v1:172.21.15.1:6790/0] mon.c 2024-09-13T04:56:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T04:56:57.343 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T04:56:57.349 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8850da77-4384-4335-9559-af2e805c4f72... 2024-09-13T04:56:57.349 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T04:56:57.401 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T04:56:57.431 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-09-13T04:56:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2024-09-13T04:56:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T04:56:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2024-09-13T04:56:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.144" 2024-09-13T04:56:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: mon client directed command retry = 5 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on pool no app = False 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: ms inject socket failures = 5000 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 8850da77-4384-4335-9559-af2e805c4f72 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 600 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: osd mclock profile = high_recovery_ops 2024-09-13T04:56:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2024-09-13T04:56:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2024-09-13T04:56:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2024-09-13T04:56:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2024-09-13T04:56:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2024-09-13T04:56:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2024-09-13T04:56:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore compression algorithm = zlib 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore compression mode = aggressive 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 1/20 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 1/20 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 20 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 4/10 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: osd blocked scrub grace period = 3600 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: osd max markdown count = 1000 2024-09-13T04:56:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: osd shutdown pgref assert = True 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-09-13T04:56:57.439 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T04:56:57.440 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.441 INFO:teuthology.orchestra.run.smithi001.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T04:56:57.441 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T04:56:57.441 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T04:56:57.441 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.441 INFO:teuthology.orchestra.run.smithi001.stdout: # 1m isn't quite enough 2024-09-13T04:56:57.441 INFO:teuthology.orchestra.run.smithi001.stdout: mon_down_mkfs_grace = 2m 2024-09-13T04:56:57.441 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.441 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_filestore_osds = false 2024-09-13T04:56:57.442 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:56:57.442 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2024-09-13T04:56:57.442 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2024-09-13T04:56:57.442 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2024-09-13T04:56:57.442 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2024-09-13T04:56:57.442 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T04:56:57.443 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T04:56:57.443 INFO:teuthology.orchestra.run.smithi001.stdout: rbd cache = True 2024-09-13T04:56:57.443 INFO:teuthology.orchestra.run.smithi001.stdout: rbd cache policy = writeback 2024-09-13T04:56:57.443 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2024-09-13T04:56:57.443 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2024-09-13T04:56:57.444 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.446 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-09-13T04:56:57.447 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.448 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.144" 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: mon client directed command retry = 5 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on pool no app = False 2024-09-13T04:56:57.449 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject socket failures = 5000 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 8850da77-4384-4335-9559-af2e805c4f72 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-09-13T04:56:57.450 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.451 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore compression algorithm = zlib 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore compression mode = aggressive 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-09-13T04:56:57.452 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: osd blocked scrub grace period = 3600 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: osd max markdown count = 1000 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: osd shutdown pgref assert = True 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-09-13T04:56:57.453 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-09-13T04:56:57.454 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.455 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: rbd cache = True 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout: rbd cache policy = writeback 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-09-13T04:56:57.456 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-09-13T04:56:57.457 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-09-13T04:56:57.457 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T04:56:57.457 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T04:56:57.542 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T04:56:57.542 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:57.542 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T04:56:57.547 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:57.547 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T04:56:57.593 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T04:56:57.594 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:57.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T04:56:57.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T04:56:57.659 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:56:57.659 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:56:57.710 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T04:56:57.710 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:56:57.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T04:56:57.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T04:56:57.733 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:56:57.733 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:56:57.781 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T04:56:57.781 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T04:56:57.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T04:56:57.827 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T04:56:57.830 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T04:56:57.873 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T04:56:57.885 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T04:56:57.885 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T04:56:57.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T04:56:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T04:56:57.931 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T04:56:57.966 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T04:56:57.976 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T04:56:57.977 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T04:56:57.977 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T04:56:57.991 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T04:56:57.991 INFO:tasks.ceph:role: osd.0 2024-09-13T04:56:57.991 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T04:56:57.991 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T04:56:58.045 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T04:56:58.045 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T04:56:58.045 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T04:56:58.045 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T04:56:58.045 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T04:56:58.046 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-09-13T04:56:58.046 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T04:56:58.046 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T04:56:58.046 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T04:56:58.046 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T04:56:58.068 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-09-13T04:56:58.070 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-09-13T04:56:58.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T04:56:58.130 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T04:56:58.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:56:58.184 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T04:56:58.184 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T04:56:58.236 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T04:56:58.236 INFO:tasks.ceph:role: osd.1 2024-09-13T04:56:58.236 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T04:56:58.236 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T04:56:58.289 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T04:56:58.289 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T04:56:58.289 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T04:56:58.289 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T04:56:58.290 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T04:56:58.290 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-09-13T04:56:58.290 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T04:56:58.290 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T04:56:58.290 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T04:56:58.290 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T04:56:58.314 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-09-13T04:56:58.316 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-09-13T04:56:58.316 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T04:56:58.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T04:56:58.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:56:58.430 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T04:56:58.430 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T04:56:58.484 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T04:56:58.484 INFO:tasks.ceph:role: osd.2 2024-09-13T04:56:58.485 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T04:56:58.485 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T04:56:58.541 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T04:56:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T04:56:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T04:56:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T04:56:58.542 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T04:56:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-09-13T04:56:58.542 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T04:56:58.542 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T04:56:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T04:56:58.542 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T04:56:58.565 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-09-13T04:56:58.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-09-13T04:56:58.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T04:56:58.628 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T04:56:58.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:56:58.679 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T04:56:58.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T04:56:58.733 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T04:56:58.733 INFO:tasks.ceph:role: osd.3 2024-09-13T04:56:58.733 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T04:56:58.734 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T04:56:58.789 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T04:56:58.789 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T04:56:58.789 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T04:56:58.790 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T04:56:58.790 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T04:56:58.790 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-09-13T04:56:58.790 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T04:56:58.790 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T04:56:58.790 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T04:56:58.790 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T04:56:58.812 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-09-13T04:56:58.814 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-09-13T04:56:58.815 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T04:56:58.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T04:56:58.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:56:58.926 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T04:56:58.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:56:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:56:59.045+0000 7f1ebc6497c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T04:56:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:56:59.045+0000 7f1ebc6497c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T04:56:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:56:59.045+0000 7f1ebc6497c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T04:56:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:56:59.045+0000 7f1ebc6497c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T04:56:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:56:59.045+0000 7f1ebc6497c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T04:56:59.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:56:59.045+0000 7f1ebc6497c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T04:57:02.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T04:57:02.060 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:02.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:02.141+0000 7fef301037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T04:57:02.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:02.141+0000 7fef301037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T04:57:02.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:02.141+0000 7fef301037c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T04:57:02.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:02.141+0000 7fef301037c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T04:57:02.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:02.141+0000 7fef301037c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T04:57:02.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:02.145+0000 7fef301037c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T04:57:05.124 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T04:57:05.179 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:05.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:05.257+0000 7f97d92787c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T04:57:05.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:05.257+0000 7f97d92787c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T04:57:05.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:05.257+0000 7f97d92787c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T04:57:05.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:05.257+0000 7f97d92787c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T04:57:05.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:05.257+0000 7f97d92787c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T04:57:05.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:05.261+0000 7f97d92787c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T04:57:08.239 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T04:57:08.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:08.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:08.377+0000 7fd71a3447c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T04:57:08.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:08.377+0000 7fd71a3447c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T04:57:08.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:08.377+0000 7fd71a3447c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T04:57:08.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:08.377+0000 7fd71a3447c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T04:57:08.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:08.377+0000 7fd71a3447c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T04:57:08.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-13T04:57:08.377+0000 7fd71a3447c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T04:57:11.349 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T04:57:11.406 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T04:57:11.422 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T04:57:11.422 INFO:tasks.ceph:role: osd.4 2024-09-13T04:57:11.422 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T04:57:11.422 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T04:57:11.488 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T04:57:11.489 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T04:57:11.974 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-13T04:57:11.976 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-13T04:57:11.976 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T04:57:12.140 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T04:57:12.151 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T04:57:12.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:57:12.152 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T04:57:12.207 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T04:57:12.207 INFO:tasks.ceph:role: osd.5 2024-09-13T04:57:12.207 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T04:57:12.208 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T04:57:12.279 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T04:57:12.719 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-13T04:57:12.721 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-13T04:57:12.721 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T04:57:12.815 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T04:57:12.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:57:12.827 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T04:57:12.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T04:57:12.881 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T04:57:12.881 INFO:tasks.ceph:role: osd.6 2024-09-13T04:57:12.881 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T04:57:12.881 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T04:57:12.942 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T04:57:12.942 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T04:57:12.942 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T04:57:12.943 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T04:57:12.943 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T04:57:12.943 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-13T04:57:12.943 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T04:57:12.943 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T04:57:12.943 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T04:57:12.943 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T04:57:13.390 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-13T04:57:13.391 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-13T04:57:13.392 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T04:57:13.476 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T04:57:13.529 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T04:57:13.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:57:13.530 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T04:57:13.584 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T04:57:13.584 INFO:tasks.ceph:role: osd.7 2024-09-13T04:57:13.584 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T04:57:13.584 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T04:57:13.648 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T04:57:13.649 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T04:57:14.150 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-09-13T04:57:14.152 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-09-13T04:57:14.152 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T04:57:14.301 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T04:57:14.311 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T04:57:14.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:57:14.312 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:14.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:14.417+0000 7fd0b757e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T04:57:14.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:14.417+0000 7fd0b757e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T04:57:14.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:14.417+0000 7fd0b757e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T04:57:14.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:14.417+0000 7fd0b757e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T04:57:14.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:14.417+0000 7fd0b757e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T04:57:14.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:14.425+0000 7fd0b757e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T04:57:17.413 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T04:57:17.427 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:17.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:17.501+0000 7f6f9117c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T04:57:17.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:17.501+0000 7f6f9117c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T04:57:17.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:17.501+0000 7f6f9117c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T04:57:17.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:17.501+0000 7f6f9117c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T04:57:17.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:17.501+0000 7f6f9117c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T04:57:17.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:17.501+0000 7f6f9117c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T04:57:20.498 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T04:57:20.556 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:20.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:20.637+0000 7f27285dd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T04:57:20.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:20.637+0000 7f27285dd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T04:57:20.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:20.637+0000 7f27285dd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T04:57:20.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:20.637+0000 7f27285dd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T04:57:20.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:20.637+0000 7f27285dd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T04:57:20.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:20.637+0000 7f27285dd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T04:57:23.632 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T04:57:23.692 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:23.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:23.773+0000 7f50a7b8e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T04:57:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:23.773+0000 7f50a7b8e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T04:57:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:23.773+0000 7f50a7b8e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T04:57:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:23.773+0000 7f50a7b8e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T04:57:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:23.773+0000 7f50a7b8e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T04:57:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:23.773+0000 7f50a7b8e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T04:57:26.750 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T04:57:26.804 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T04:57:26.805 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:26.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T04:57:26.820 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:26.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T04:57:26.873 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:26.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T04:57:26.929 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:26.929 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T04:57:26.985 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:26.985 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T04:57:27.040 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:27.040 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T04:57:27.053 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:27.054 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T04:57:27.107 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:27.107 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T04:57:27.168 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:27.168 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T04:57:27.223 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:27.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T04:57:27.276 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:27.276 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T04:57:27.282 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:27.282 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T04:57:27.324 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T04:57:27.325 DEBUG:teuthology.orchestra.run.smithi001:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T04:57:27.327 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T04:57:27.370 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.y] 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAZxuNmlFhiMRAAENAuXiODv22r78jYN7doZw== 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.0] 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAbxuNmsMPUAhAAYmZUfPXYfGgVyY6Ii2ksNw== 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.1] 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAexuNmlMmvCBAAW3Y0mVolGEAqnN129mVdeQ== 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.2] 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAhxuNmzu6XDxAA+01A+3ZE00DI8FX5j01UpA== 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.3] 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAkxuNmno+PFhAAU1lIsyXkg8n4t6rYmwJb6g== 2024-09-13T04:57:27.371 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.x] 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAZxuNmbJQhNBAAvEVCJS/uXkQuuqQWZxrBeg== 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.4] 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAqxuNmZawoGRAATuxNc1tF+12WpJSPvOAljg== 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.5] 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAtxuNmb1c9HhAA7P+fwSZXOv62tEOJgmS3EA== 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.6] 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAwxuNmzDIfJhAAP0+reID86Fn2gXfxzTctlg== 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.7] 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAzxuNmpyszLhAAClm8jcQ3KQhbNP/fGCwM6Q== 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAZxuNmsYvqNhAAxcH2tc2ltC5vl6vt7mQU9g== 2024-09-13T04:57:27.372 INFO:teuthology.orchestra.run.smithi001.stdout:[client.1] 2024-09-13T04:57:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQAZxuNmYtSfORAAtwiq02vkgkse94Vxk5VV8w== 2024-09-13T04:57:27.378 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.y] 2024-09-13T04:57:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAZxuNmlFhiMRAAENAuXiODv22r78jYN7doZw== 2024-09-13T04:57:27.378 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2024-09-13T04:57:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAbxuNmsMPUAhAAYmZUfPXYfGgVyY6Ii2ksNw== 2024-09-13T04:57:27.378 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAexuNmlMmvCBAAW3Y0mVolGEAqnN129mVdeQ== 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAhxuNmzu6XDxAA+01A+3ZE00DI8FX5j01UpA== 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAkxuNmno+PFhAAU1lIsyXkg8n4t6rYmwJb6g== 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAZxuNmbJQhNBAAvEVCJS/uXkQuuqQWZxrBeg== 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAqxuNmZawoGRAATuxNc1tF+12WpJSPvOAljg== 2024-09-13T04:57:27.379 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAtxuNmb1c9HhAA7P+fwSZXOv62tEOJgmS3EA== 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.6] 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAwxuNmzDIfJhAAP0+reID86Fn2gXfxzTctlg== 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.7] 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAzxuNmpyszLhAAClm8jcQ3KQhbNP/fGCwM6Q== 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAZxuNmsYvqNhAAxcH2tc2ltC5vl6vt7mQU9g== 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2024-09-13T04:57:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAZxuNmYtSfORAAtwiq02vkgkse94Vxk5VV8w== 2024-09-13T04:57:27.381 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T04:57:27.418 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T04:57:27.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.471 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.552 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.554 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.638 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.719 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.722 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T04:57:27.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T04:57:27.883 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.885 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.967 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:27.970 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:28.056 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:28.058 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:28.144 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:28.146 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T04:57:28.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T04:57:28.230 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T04:57:28.319 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T04:57:28.321 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T04:57:28.369 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T04:57:28.369 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T04:57:28.382 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T04:57:28.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T04:57:28.487 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T04:57:28.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T04:57:28.632 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T04:57:28.643 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T04:57:28.657 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T04:57:28.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T04:57:28.762 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:28.765 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T04:57:28.808 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T04:57:28.809 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T04:57:28.809 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T04:57:28.811 INFO:tasks.ceph.mon.a:Started 2024-09-13T04:57:28.811 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T04:57:28.811 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T04:57:28.814 INFO:tasks.ceph.mon.c:Started 2024-09-13T04:57:28.814 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T04:57:28.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T04:57:28.853 INFO:tasks.ceph.mon.b:Started 2024-09-13T04:57:28.853 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T04:57:28.854 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T04:57:28.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T04:57:28.856 INFO:tasks.ceph.mgr.y:Started 2024-09-13T04:57:28.856 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T04:57:28.856 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T04:57:28.858 INFO:tasks.ceph.mgr.x:Started 2024-09-13T04:57:28.859 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T04:57:28.859 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T04:57:34.352 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-09-13T04:57:34.372 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T04:57:34.373 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T04:57:34.373 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:34.373 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T04:57:34.386 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:34.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T04:57:34.440 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:34.440 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T04:57:34.468 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:34.465+0000 7f262cb28180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T04:57:34.479 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-13T04:57:34.479 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T04:57:34.513 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:34.512+0000 7f564982b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T04:57:34.532 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:34.532 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T04:57:34.545 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:34.545 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T04:57:34.599 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:34.600 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T04:57:34.656 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-13T04:57:34.656 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T04:57:34.711 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 2fda29ca-3c6d-47ae-9244-8ccdf55f89f6 0 2024-09-13T04:57:34.933 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:34.933+0000 7f262cb28180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T04:57:35.037 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:35.036+0000 7f564982b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T04:57:35.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-13T04:57:35.092+0000 7f1d577fe640 0 --2- 172.21.15.144:0/789908210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d5813a110 0x7f1d5813acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T04:57:35.135 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:35.133+0000 7f262cb28180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T04:57:35.214 INFO:teuthology.orchestra.run.smithi144.stdout:0 2024-09-13T04:57:35.235 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 9ca6e867-faf0-4295-b8eb-4c738dd26cb9 1 2024-09-13T04:57:35.267 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:35.264+0000 7f564982b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T04:57:35.440 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-09-13T04:57:35.457 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new d8159400-8662-4567-8714-b48c2d8f3ad2 2 2024-09-13T04:57:35.704 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:35.701+0000 7f262cb28180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T04:57:35.709 INFO:teuthology.orchestra.run.smithi144.stdout:2 2024-09-13T04:57:35.726 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 13a3ea70-fa37-41d8-bae7-0203952b02cb 3 2024-09-13T04:57:35.760 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:35.757+0000 7f262cb28180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T04:57:35.815 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:35.813+0000 7f262cb28180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T04:57:35.828 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:35.828+0000 7f564982b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T04:57:35.884 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:35.884+0000 7f564982b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T04:57:35.896 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:35.893+0000 7f262cb28180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T04:57:35.926 INFO:teuthology.orchestra.run.smithi144.stdout:3 2024-09-13T04:57:35.939 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:35.936+0000 7f564982b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T04:57:35.943 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new e896e6b5-c54a-405f-ab69-2a5b7b7a153e 4 2024-09-13T04:57:35.951 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:35.949+0000 7f262cb28180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T04:57:36.018 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:36.016+0000 7f564982b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T04:57:36.031 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:36.029+0000 7f262cb28180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T04:57:36.073 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:36.072+0000 7f564982b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T04:57:36.084 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:36.085+0000 7f262cb28180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T04:57:36.136 INFO:teuthology.orchestra.run.smithi144.stdout:4 2024-09-13T04:57:36.153 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 7de23072-7eda-43d6-9b92-d608ef82c6fa 5 2024-09-13T04:57:36.153 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:36.152+0000 7f564982b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T04:57:36.206 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:36.204+0000 7f564982b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T04:57:36.242 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:36.241+0000 7f262cb28180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T04:57:36.305 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:36.305+0000 7f262cb28180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T04:57:36.355 INFO:teuthology.orchestra.run.smithi144.stdout:5 2024-09-13T04:57:36.362 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:36.360+0000 7f564982b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T04:57:36.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 8719fad9-81a0-46be-a546-db83bcace33b 6 2024-09-13T04:57:36.425 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:36.424+0000 7f564982b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T04:57:36.562 INFO:teuthology.orchestra.run.smithi144.stdout:6 2024-09-13T04:57:36.578 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 8bf0a351-e58e-4393-a6ce-fe70acf362aa 7 2024-09-13T04:57:36.819 INFO:teuthology.orchestra.run.smithi144.stdout:7 2024-09-13T04:57:36.835 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T04:57:36.835 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T04:57:36.837 INFO:tasks.ceph.osd.0:Started 2024-09-13T04:57:36.837 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T04:57:36.837 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T04:57:36.839 INFO:tasks.ceph.osd.1:Started 2024-09-13T04:57:36.840 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T04:57:36.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T04:57:36.842 INFO:tasks.ceph.osd.2:Started 2024-09-13T04:57:36.842 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T04:57:36.842 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T04:57:36.844 INFO:tasks.ceph.osd.3:Started 2024-09-13T04:57:36.844 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T04:57:36.844 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T04:57:36.846 INFO:tasks.ceph.osd.4:Started 2024-09-13T04:57:36.846 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T04:57:36.847 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T04:57:36.848 INFO:tasks.ceph.osd.5:Started 2024-09-13T04:57:36.848 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T04:57:36.848 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T04:57:36.850 INFO:tasks.ceph.osd.6:Started 2024-09-13T04:57:36.850 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T04:57:36.850 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T04:57:36.852 INFO:tasks.ceph.osd.7:Started 2024-09-13T04:57:36.853 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T04:57:36.941 INFO:tasks.ceph.mgr.y.smithi001.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T04:57:36.941 INFO:tasks.ceph.mgr.y.smithi001.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T04:57:36.942 INFO:tasks.ceph.mgr.y.smithi001.stderr: from numpy import show_config as show_numpy_config 2024-09-13T04:57:37.001 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:37.001+0000 7f262cb28180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T04:57:37.032 INFO:tasks.ceph.mgr.x.smithi144.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T04:57:37.033 INFO:tasks.ceph.mgr.x.smithi144.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T04:57:37.033 INFO:tasks.ceph.mgr.x.smithi144.stderr: from numpy import show_config as show_numpy_config 2024-09-13T04:57:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:57:37.078 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T04:57:36.815703+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T04:57:36.815703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T04:57:37.094 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T04:57:37.094 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T04:57:37.107 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:37.104+0000 7f564982b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T04:57:37.158 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:37.157+0000 7f262cb28180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T04:57:37.259 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:37.256+0000 7f564982b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T04:57:37.317 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:37.317+0000 7f262cb28180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T04:57:37.383 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:37.381+0000 7f262cb28180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T04:57:37.433 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:37.432+0000 7f564982b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T04:57:37.454 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:57:37.453+0000 7f546a0987c0 -1 Falling back to public interface 2024-09-13T04:57:37.457 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:57:37.456+0000 7ff2e78027c0 -1 Falling back to public interface 2024-09-13T04:57:37.461 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:57:37.460+0000 7f9a8453b7c0 -1 Falling back to public interface 2024-09-13T04:57:37.469 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:57:37.468+0000 7f5a136c97c0 -1 Falling back to public interface 2024-09-13T04:57:37.469 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:57:37.468+0000 7fb4ea5487c0 -1 Falling back to public interface 2024-09-13T04:57:37.469 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:57:37.469+0000 7ffa4efe67c0 -1 Falling back to public interface 2024-09-13T04:57:37.481 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:57:37.481+0000 7faf85c317c0 -1 Falling back to public interface 2024-09-13T04:57:37.481 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:57:37.481+0000 7f92c5e407c0 -1 Falling back to public interface 2024-09-13T04:57:37.505 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:37.504+0000 7f564982b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T04:57:37.529 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:37.529+0000 7f262cb28180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T04:57:37.651 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:37.648+0000 7f564982b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T04:57:37.901 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T04:57:38.081 INFO:teuthology.misc.health.smithi001.stdout: 2024-09-13T04:57:38.082 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":10,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T04:57:36.815703+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T04:57:36.815703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T04:57:38.097 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T04:57:38.160 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:38.157+0000 7f262cb28180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T04:57:38.238 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:38.237+0000 7f262cb28180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T04:57:38.271 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:38.268+0000 7f564982b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T04:57:38.346 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:38.344+0000 7f564982b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T04:57:38.427 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:38.425+0000 7f262cb28180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T04:57:38.503 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:38.501+0000 7f262cb28180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T04:57:38.525 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:38.524+0000 7f564982b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T04:57:38.596 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:38.596+0000 7f564982b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T04:57:38.806 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:38.805+0000 7f262cb28180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T04:57:38.870 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:38.869+0000 7f262cb28180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T04:57:38.882 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:38.880+0000 7f564982b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T04:57:38.946 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-09-13T04:57:38.944+0000 7f564982b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T04:57:39.918 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:39.917+0000 7f25d40da640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T04:57:39.919 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-09-13T04:57:39.917+0000 7f25d40da640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T04:57:40.842 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:57:40.840+0000 7f5a136c97c0 -1 osd.5 0 log_to_monitors true 2024-09-13T04:57:40.964 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:57:40.964+0000 7ff2e78027c0 -1 osd.4 0 log_to_monitors true 2024-09-13T04:57:41.064 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:57:41.064+0000 7f9a8453b7c0 -1 osd.7 0 log_to_monitors true 2024-09-13T04:57:41.112 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:57:41.109+0000 7f92c5e407c0 -1 osd.0 0 log_to_monitors true 2024-09-13T04:57:41.264 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:57:41.264+0000 7fb4ea5487c0 -1 osd.6 0 log_to_monitors true 2024-09-13T04:57:41.383 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:57:41.381+0000 7f546a0987c0 -1 osd.2 0 log_to_monitors true 2024-09-13T04:57:41.384 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:57:41.381+0000 7faf85c317c0 -1 osd.1 0 log_to_monitors true 2024-09-13T04:57:41.390 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:57:41.389+0000 7ffa4efe67c0 -1 osd.3 0 log_to_monitors true 2024-09-13T04:57:44.904 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T04:57:44.921 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:57:44.920+0000 7f5a0fe85640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T04:57:44.961 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:57:44.960+0000 7f5a074a1640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T04:57:45.263 INFO:teuthology.misc.health.smithi001.stdout: 2024-09-13T04:57:45.264 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":13,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T04:57:42.911534+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T04:57:36.815703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T04:57:45.287 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T04:57:46.078 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:57:46.076+0000 7fb4e64ed640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T04:57:46.095 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:57:46.092+0000 7f9a804e0640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T04:57:46.105 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:57:46.104+0000 7fb4de320640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T04:57:46.112 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:57:46.112+0000 7f9a78313640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T04:57:46.141 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:57:46.140+0000 7ff2e37a7640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T04:57:46.185 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:57:46.184+0000 7ff2db5da640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T04:57:46.367 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:57:46.365+0000 7f546603d640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T04:57:46.382 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:57:46.381+0000 7ffa4af8b640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T04:57:46.389 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:57:46.389+0000 7f545de70640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T04:57:46.405 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:57:46.405+0000 7ffa42dbe640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T04:57:46.461 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:57:46.461+0000 7faf823ed640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T04:57:46.466 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:57:46.465+0000 7f92c1de5640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T04:57:46.481 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:57:46.481+0000 7faf79a09640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T04:57:46.489 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:57:46.489+0000 7f92b9c18640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T04:57:52.095 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T04:57:52.397 INFO:teuthology.misc.health.smithi001.stdout: 2024-09-13T04:57:52.397 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":19,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T04:57:51.123155+0000","last_up_change":"2024-09-13T04:57:47.098606+0000","last_in_change":"2024-09-13T04:57:36.815703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T04:57:48.984599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6820","nonce":2346968854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6824","nonce":2346968854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6831","nonce":2346968854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6828","nonce":2346968854}]},"public_addr":"172.21.15.1:6820/2346968854","cluster_addr":"172.21.15.1:6824/2346968854","heartbeat_back_addr":"172.21.15.1:6831/2346968854","heartbeat_front_addr":"172.21.15.1:6828/2346968854","state":["exists","up"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6817","nonce":466532899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6821","nonce":466532899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6829","nonce":466532899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6825","nonce":466532899}]},"public_addr":"172.21.15.1:6817/466532899","cluster_addr":"172.21.15.1:6821/466532899","heartbeat_back_addr":"172.21.15.1:6829/466532899","heartbeat_front_addr":"172.21.15.1:6825/466532899","state":["exists","up"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6801","nonce":343712578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6803","nonce":343712578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6807","nonce":343712578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6805","nonce":343712578}]},"public_addr":"172.21.15.1:6801/343712578","cluster_addr":"172.21.15.1:6803/343712578","heartbeat_back_addr":"172.21.15.1:6807/343712578","heartbeat_front_addr":"172.21.15.1:6805/343712578","state":["exists","up"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6809","nonce":2021827348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6811","nonce":2021827348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6815","nonce":2021827348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6813","nonce":2021827348}]},"public_addr":"172.21.15.1:6809/2021827348","cluster_addr":"172.21.15.1:6811/2021827348","heartbeat_back_addr":"172.21.15.1:6815/2021827348","heartbeat_front_addr":"172.21.15.1:6813/2021827348","state":["exists","up"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6801","nonce":2183053562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6803","nonce":2183053562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6807","nonce":2183053562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6805","nonce":2183053562}]},"public_addr":"172.21.15.144:6801/2183053562","cluster_addr":"172.21.15.144:6803/2183053562","heartbeat_back_addr":"172.21.15.144:6807/2183053562","heartbeat_front_addr":"172.21.15.144:6805/2183053562","state":["exists","up"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6817","nonce":2765053006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6821","nonce":2765053006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6828","nonce":2765053006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6824","nonce":2765053006}]},"public_addr":"172.21.15.144:6817/2765053006","cluster_addr":"172.21.15.144:6821/2765053006","heartbeat_back_addr":"172.21.15.144:6828/2765053006","heartbeat_front_addr":"172.21.15.144:6824/2765053006","state":["exists","up"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6820","nonce":2721913793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6825","nonce":2721913793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6831","nonce":2721913793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6827","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6829","nonce":2721913793}]},"public_addr":"172.21.15.144:6820/2721913793","cluster_addr":"172.21.15.144:6825/2721913793","heartbeat_back_addr":"172.21.15.144:6831/2721913793","heartbeat_front_addr":"172.21.15.144:6829/2721913793","state":["exists","up"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6809","nonce":4202513770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6811","nonce":4202513770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6815","nonce":4202513770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6813","nonce":4202513770}]},"public_addr":"172.21.15.144:6809/4202513770","cluster_addr":"172.21.15.144:6811/4202513770","heartbeat_back_addr":"172.21.15.144:6815/4202513770","heartbeat_front_addr":"172.21.15.144:6813/4202513770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.092681+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.414226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.368701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.370707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.980085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.874193+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.219724+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.114987+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T04:57:52.415 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T04:57:52.415 INFO:tasks.ceph:Creating RBD pool 2024-09-13T04:57:52.415 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T04:57:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'rbd' created 2024-09-13T04:57:53.170 DEBUG:teuthology.orchestra.run.smithi001:> rbd --cluster ceph pool init rbd 2024-09-13T04:57:56.175 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T04:57:56.176 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T04:57:56.176 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T04:57:56.176 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T04:57:56.177 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T04:57:56.533 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:57:56.549 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2102693671},{"type":"v1","addr":"172.21.15.1:6833","nonce":2102693671}]},"active_addr":"172.21.15.1:6833/2102693671","active_change":"2024-09-13T04:57:38.892323+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":634568643}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":3941865217}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1624186499}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1778527407}]}]} 2024-09-13T04:57:56.554 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T04:57:56.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T04:57:56.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T04:57:56.835 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:57:56.835 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T04:57:56.147154+0000","last_up_change":"2024-09-13T04:57:47.098606+0000","last_in_change":"2024-09-13T04:57:36.815703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T04:57:48.984599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T04:57:52.756781+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6820","nonce":2346968854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6824","nonce":2346968854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6831","nonce":2346968854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6828","nonce":2346968854}]},"public_addr":"172.21.15.1:6820/2346968854","cluster_addr":"172.21.15.1:6824/2346968854","heartbeat_back_addr":"172.21.15.1:6831/2346968854","heartbeat_front_addr":"172.21.15.1:6828/2346968854","state":["exists","up"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6817","nonce":466532899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6821","nonce":466532899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6829","nonce":466532899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6825","nonce":466532899}]},"public_addr":"172.21.15.1:6817/466532899","cluster_addr":"172.21.15.1:6821/466532899","heartbeat_back_addr":"172.21.15.1:6829/466532899","heartbeat_front_addr":"172.21.15.1:6825/466532899","state":["exists","up"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6801","nonce":343712578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6803","nonce":343712578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6807","nonce":343712578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6805","nonce":343712578}]},"public_addr":"172.21.15.1:6801/343712578","cluster_addr":"172.21.15.1:6803/343712578","heartbeat_back_addr":"172.21.15.1:6807/343712578","heartbeat_front_addr":"172.21.15.1:6805/343712578","state":["exists","up"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6809","nonce":2021827348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6811","nonce":2021827348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6815","nonce":2021827348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6813","nonce":2021827348}]},"public_addr":"172.21.15.1:6809/2021827348","cluster_addr":"172.21.15.1:6811/2021827348","heartbeat_back_addr":"172.21.15.1:6815/2021827348","heartbeat_front_addr":"172.21.15.1:6813/2021827348","state":["exists","up"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6801","nonce":2183053562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6803","nonce":2183053562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6807","nonce":2183053562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6805","nonce":2183053562}]},"public_addr":"172.21.15.144:6801/2183053562","cluster_addr":"172.21.15.144:6803/2183053562","heartbeat_back_addr":"172.21.15.144:6807/2183053562","heartbeat_front_addr":"172.21.15.144:6805/2183053562","state":["exists","up"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6817","nonce":2765053006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6821","nonce":2765053006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6828","nonce":2765053006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6824","nonce":2765053006}]},"public_addr":"172.21.15.144:6817/2765053006","cluster_addr":"172.21.15.144:6821/2765053006","heartbeat_back_addr":"172.21.15.144:6828/2765053006","heartbeat_front_addr":"172.21.15.144:6824/2765053006","state":["exists","up"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6820","nonce":2721913793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6825","nonce":2721913793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6831","nonce":2721913793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6827","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6829","nonce":2721913793}]},"public_addr":"172.21.15.144:6820/2721913793","cluster_addr":"172.21.15.144:6825/2721913793","heartbeat_back_addr":"172.21.15.144:6831/2721913793","heartbeat_front_addr":"172.21.15.144:6829/2721913793","state":["exists","up"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6809","nonce":4202513770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6811","nonce":4202513770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6815","nonce":4202513770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6813","nonce":4202513770}]},"public_addr":"172.21.15.144:6809/4202513770","cluster_addr":"172.21.15.144:6811/4202513770","heartbeat_back_addr":"172.21.15.144:6815/4202513770","heartbeat_front_addr":"172.21.15.144:6813/4202513770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.092681+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.414226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.368701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.370707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.980085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.874193+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.219724+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.114987+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T04:57:56.853 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T04:57:56.853 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T04:57:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:57:57.145 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T04:57:56.147154+0000","last_up_change":"2024-09-13T04:57:47.098606+0000","last_in_change":"2024-09-13T04:57:36.815703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T04:57:48.984599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T04:57:52.756781+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6820","nonce":2346968854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6824","nonce":2346968854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6831","nonce":2346968854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6828","nonce":2346968854}]},"public_addr":"172.21.15.1:6820/2346968854","cluster_addr":"172.21.15.1:6824/2346968854","heartbeat_back_addr":"172.21.15.1:6831/2346968854","heartbeat_front_addr":"172.21.15.1:6828/2346968854","state":["exists","up"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6817","nonce":466532899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6821","nonce":466532899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6829","nonce":466532899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6825","nonce":466532899}]},"public_addr":"172.21.15.1:6817/466532899","cluster_addr":"172.21.15.1:6821/466532899","heartbeat_back_addr":"172.21.15.1:6829/466532899","heartbeat_front_addr":"172.21.15.1:6825/466532899","state":["exists","up"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6801","nonce":343712578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6803","nonce":343712578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6807","nonce":343712578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6805","nonce":343712578}]},"public_addr":"172.21.15.1:6801/343712578","cluster_addr":"172.21.15.1:6803/343712578","heartbeat_back_addr":"172.21.15.1:6807/343712578","heartbeat_front_addr":"172.21.15.1:6805/343712578","state":["exists","up"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6809","nonce":2021827348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6811","nonce":2021827348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6815","nonce":2021827348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6813","nonce":2021827348}]},"public_addr":"172.21.15.1:6809/2021827348","cluster_addr":"172.21.15.1:6811/2021827348","heartbeat_back_addr":"172.21.15.1:6815/2021827348","heartbeat_front_addr":"172.21.15.1:6813/2021827348","state":["exists","up"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6801","nonce":2183053562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6803","nonce":2183053562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6807","nonce":2183053562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6805","nonce":2183053562}]},"public_addr":"172.21.15.144:6801/2183053562","cluster_addr":"172.21.15.144:6803/2183053562","heartbeat_back_addr":"172.21.15.144:6807/2183053562","heartbeat_front_addr":"172.21.15.144:6805/2183053562","state":["exists","up"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6817","nonce":2765053006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6821","nonce":2765053006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6828","nonce":2765053006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6824","nonce":2765053006}]},"public_addr":"172.21.15.144:6817/2765053006","cluster_addr":"172.21.15.144:6821/2765053006","heartbeat_back_addr":"172.21.15.144:6828/2765053006","heartbeat_front_addr":"172.21.15.144:6824/2765053006","state":["exists","up"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6820","nonce":2721913793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6825","nonce":2721913793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6831","nonce":2721913793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6827","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6829","nonce":2721913793}]},"public_addr":"172.21.15.144:6820/2721913793","cluster_addr":"172.21.15.144:6825/2721913793","heartbeat_back_addr":"172.21.15.144:6831/2721913793","heartbeat_front_addr":"172.21.15.144:6829/2721913793","state":["exists","up"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6809","nonce":4202513770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6811","nonce":4202513770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6815","nonce":4202513770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6813","nonce":4202513770}]},"public_addr":"172.21.15.144:6809/4202513770","cluster_addr":"172.21.15.144:6811/4202513770","heartbeat_back_addr":"172.21.15.144:6815/4202513770","heartbeat_front_addr":"172.21.15.144:6813/4202513770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.092681+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.414226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.368701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.370707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.980085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.874193+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.219724+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.114987+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T04:57:57.162 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T04:57:57.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T04:57:57.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T04:57:57.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T04:57:57.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T04:57:57.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T04:57:57.165 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T04:57:57.165 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T04:57:57.413 INFO:teuthology.orchestra.run.smithi001.stdout:64424509444 2024-09-13T04:57:57.413 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T04:57:57.432 INFO:teuthology.orchestra.run.smithi001.stdout:64424509447 2024-09-13T04:57:57.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T04:57:57.433 INFO:teuthology.orchestra.run.smithi001.stdout:64424509444 2024-09-13T04:57:57.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T04:57:57.434 INFO:teuthology.orchestra.run.smithi001.stdout:64424509445 2024-09-13T04:57:57.434 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T04:57:57.439 INFO:teuthology.orchestra.run.smithi001.stdout:64424509444 2024-09-13T04:57:57.440 INFO:teuthology.orchestra.run.smithi001.stdout:60129542153 2024-09-13T04:57:57.441 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T04:57:57.441 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T04:57:57.442 INFO:teuthology.orchestra.run.smithi001.stdout:64424509444 2024-09-13T04:57:57.442 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T04:57:57.447 INFO:teuthology.orchestra.run.smithi001.stdout:64424509444 2024-09-13T04:57:57.447 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T04:57:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:64424509443 2024-09-13T04:57:57.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.2 2024-09-13T04:57:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:64424509444 2024-09-13T04:57:57.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.4 2024-09-13T04:57:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:60129542151 2024-09-13T04:57:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:64424509443 2024-09-13T04:57:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:64424509443 2024-09-13T04:57:57.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.0 2024-09-13T04:57:57.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542151 for osd.5 2024-09-13T04:57:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:64424509446 2024-09-13T04:57:58.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.1 2024-09-13T04:57:58.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-13T04:57:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:64424509443 2024-09-13T04:57:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:64424509443 2024-09-13T04:57:58.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.3 2024-09-13T04:57:58.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.6 2024-09-13T04:57:58.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T04:57:58.962 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T04:57:58.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T04:57:58.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T04:57:59.006 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T04:57:59.010 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T04:57:59.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T04:57:59.039 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T04:57:59.420 INFO:teuthology.orchestra.run.smithi001.stdout:64424509445 2024-09-13T04:57:59.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.2 2024-09-13T04:57:59.443 DEBUG:teuthology.parallel:result is None 2024-09-13T04:57:59.475 INFO:teuthology.orchestra.run.smithi001.stdout:64424509446 2024-09-13T04:57:59.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.4 2024-09-13T04:57:59.493 DEBUG:teuthology.parallel:result is None 2024-09-13T04:57:59.540 INFO:teuthology.orchestra.run.smithi001.stdout:60129542153 2024-09-13T04:57:59.549 INFO:teuthology.orchestra.run.smithi001.stdout:64424509448 2024-09-13T04:57:59.559 INFO:teuthology.orchestra.run.smithi001.stdout:64424509445 2024-09-13T04:57:59.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.5 2024-09-13T04:57:59.560 DEBUG:teuthology.parallel:result is None 2024-09-13T04:57:59.567 INFO:teuthology.orchestra.run.smithi001.stdout:64424509445 2024-09-13T04:57:59.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.7 2024-09-13T04:57:59.569 DEBUG:teuthology.parallel:result is None 2024-09-13T04:57:59.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.1 2024-09-13T04:57:59.586 DEBUG:teuthology.parallel:result is None 2024-09-13T04:57:59.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.0 2024-09-13T04:57:59.591 DEBUG:teuthology.parallel:result is None 2024-09-13T04:57:59.605 INFO:teuthology.orchestra.run.smithi001.stdout:64424509445 2024-09-13T04:57:59.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.3 2024-09-13T04:57:59.626 DEBUG:teuthology.parallel:result is None 2024-09-13T04:57:59.628 INFO:teuthology.orchestra.run.smithi001.stdout:64424509445 2024-09-13T04:57:59.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.6 2024-09-13T04:57:59.645 DEBUG:teuthology.parallel:result is None 2024-09-13T04:57:59.645 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T04:57:59.645 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:57:59.969 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:57:59.970 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:57:59.986 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T04:57:58.909327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238616,"kb_used_data":1944,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754736104,"statfs":{"total":773094113280,"available":772849770496,"internally_reserved":0,"allocated":1990656,"data_stored":1811694,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.766582"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155829+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:57:56.155829+0000","last_peered":"2024-09-13T04:57:56.155829+0000","last_clean":"2024-09-13T04:57:56.155829+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:57:56.155829+0000","last_undegraded":"2024-09-13T04:57:56.155829+0000","last_fullsized":"2024-09-13T04:57:56.155829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.154929+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:57:56.154929+0000","last_peered":"2024-09-13T04:57:56.154929+0000","last_clean":"2024-09-13T04:57:56.154929+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:57:56.154929+0000","last_undegraded":"2024-09-13T04:57:56.154929+0000","last_fullsized":"2024-09-13T04:57:56.154929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155102+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:57:56.155102+0000","last_peered":"2024-09-13T04:57:56.155102+0000","last_clean":"2024-09-13T04:57:56.155102+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:57:56.155102+0000","last_undegraded":"2024-09-13T04:57:56.155102+0000","last_fullsized":"2024-09-13T04:57:56.155102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.499872+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:57:56.499872+0000","last_peered":"2024-09-13T04:57:56.499872+0000","last_clean":"2024-09-13T04:57:56.499872+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:57:56.499872+0000","last_undegraded":"2024-09-13T04:57:56.499872+0000","last_fullsized":"2024-09-13T04:57:56.499872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.161458+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:57:56.161458+0000","last_peered":"2024-09-13T04:57:56.161458+0000","last_clean":"2024-09-13T04:57:56.161458+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:57:56.161458+0000","last_undegraded":"2024-09-13T04:57:56.161458+0000","last_fullsized":"2024-09-13T04:57:56.161458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.500205+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:57:56.500205+0000","last_peered":"2024-09-13T04:57:56.500205+0000","last_clean":"2024-09-13T04:57:56.500205+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:57:56.500205+0000","last_undegraded":"2024-09-13T04:57:56.500205+0000","last_fullsized":"2024-09-13T04:57:56.500205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.154502+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:57:56.154502+0000","last_peered":"2024-09-13T04:57:56.154502+0000","last_clean":"2024-09-13T04:57:56.154502+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:57:56.154502+0000","last_undegraded":"2024-09-13T04:57:56.154502+0000","last_fullsized":"2024-09-13T04:57:56.154502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155387+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:57:56.155387+0000","last_peered":"2024-09-13T04:57:56.155387+0000","last_clean":"2024-09-13T04:57:56.155387+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:57:56.155387+0000","last_undegraded":"2024-09-13T04:57:56.155387+0000","last_fullsized":"2024-09-13T04:57:56.155387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T04:57:55.894891+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:57:55.894891+0000","last_peered":"2024-09-13T04:57:55.894891+0000","last_clean":"2024-09-13T04:57:55.894891+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:57:55.894891+0000","last_undegraded":"2024-09-13T04:57:55.894891+0000","last_fullsized":"2024-09-13T04:57:55.894891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":78884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":237568,"data_stored":78884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":290816,"data_stored":669233,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":290816,"data_stored":669233,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:57:59.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:58:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:58:00.296 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T04:57:58.909327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238616,"kb_used_data":1944,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754736104,"statfs":{"total":773094113280,"available":772849770496,"internally_reserved":0,"allocated":1990656,"data_stored":1811694,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.766582"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155829+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:57:56.155829+0000","last_peered":"2024-09-13T04:57:56.155829+0000","last_clean":"2024-09-13T04:57:56.155829+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:57:56.155829+0000","last_undegraded":"2024-09-13T04:57:56.155829+0000","last_fullsized":"2024-09-13T04:57:56.155829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.154929+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:57:56.154929+0000","last_peered":"2024-09-13T04:57:56.154929+0000","last_clean":"2024-09-13T04:57:56.154929+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:57:56.154929+0000","last_undegraded":"2024-09-13T04:57:56.154929+0000","last_fullsized":"2024-09-13T04:57:56.154929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155102+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:57:56.155102+0000","last_peered":"2024-09-13T04:57:56.155102+0000","last_clean":"2024-09-13T04:57:56.155102+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:57:56.155102+0000","last_undegraded":"2024-09-13T04:57:56.155102+0000","last_fullsized":"2024-09-13T04:57:56.155102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.499872+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:57:56.499872+0000","last_peered":"2024-09-13T04:57:56.499872+0000","last_clean":"2024-09-13T04:57:56.499872+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:57:56.499872+0000","last_undegraded":"2024-09-13T04:57:56.499872+0000","last_fullsized":"2024-09-13T04:57:56.499872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.161458+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:57:56.161458+0000","last_peered":"2024-09-13T04:57:56.161458+0000","last_clean":"2024-09-13T04:57:56.161458+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:57:56.161458+0000","last_undegraded":"2024-09-13T04:57:56.161458+0000","last_fullsized":"2024-09-13T04:57:56.161458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.500205+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:57:56.500205+0000","last_peered":"2024-09-13T04:57:56.500205+0000","last_clean":"2024-09-13T04:57:56.500205+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:57:56.500205+0000","last_undegraded":"2024-09-13T04:57:56.500205+0000","last_fullsized":"2024-09-13T04:57:56.500205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.154502+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:57:56.154502+0000","last_peered":"2024-09-13T04:57:56.154502+0000","last_clean":"2024-09-13T04:57:56.154502+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:57:56.154502+0000","last_undegraded":"2024-09-13T04:57:56.154502+0000","last_fullsized":"2024-09-13T04:57:56.154502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155387+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:57:56.155387+0000","last_peered":"2024-09-13T04:57:56.155387+0000","last_clean":"2024-09-13T04:57:56.155387+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:57:56.155387+0000","last_undegraded":"2024-09-13T04:57:56.155387+0000","last_fullsized":"2024-09-13T04:57:56.155387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T04:57:55.894891+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:57:55.894891+0000","last_peered":"2024-09-13T04:57:55.894891+0000","last_clean":"2024-09-13T04:57:55.894891+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:57:55.894891+0000","last_undegraded":"2024-09-13T04:57:55.894891+0000","last_fullsized":"2024-09-13T04:57:55.894891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":78884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":237568,"data_stored":78884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":290816,"data_stored":669233,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":290816,"data_stored":669233,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:58:00.297 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T04:58:00.297 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T04:58:00.297 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T04:58:00.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T04:58:00.616 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:00.616 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T04:58:00.634 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T04:58:00.634 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T04:58:00.668 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T04:58:00.668 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}} 2024-09-13T04:58:00.669 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T04:58:00.669 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T04:58:00.985 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T04:58:00.985 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T04:58:00.985 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:58:01.271 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:01.272 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:58:01.288 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T04:58:00.909900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238616,"kb_used_data":1944,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754736104,"statfs":{"total":773094113280,"available":772849770496,"internally_reserved":0,"allocated":1990656,"data_stored":1811694,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.764613"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155829+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:57:56.155829+0000","last_peered":"2024-09-13T04:57:56.155829+0000","last_clean":"2024-09-13T04:57:56.155829+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:57:56.155829+0000","last_undegraded":"2024-09-13T04:57:56.155829+0000","last_fullsized":"2024-09-13T04:57:56.155829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.154929+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:57:56.154929+0000","last_peered":"2024-09-13T04:57:56.154929+0000","last_clean":"2024-09-13T04:57:56.154929+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:57:56.154929+0000","last_undegraded":"2024-09-13T04:57:56.154929+0000","last_fullsized":"2024-09-13T04:57:56.154929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155102+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:57:56.155102+0000","last_peered":"2024-09-13T04:57:56.155102+0000","last_clean":"2024-09-13T04:57:56.155102+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:57:56.155102+0000","last_undegraded":"2024-09-13T04:57:56.155102+0000","last_fullsized":"2024-09-13T04:57:56.155102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.499872+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:57:56.499872+0000","last_peered":"2024-09-13T04:57:56.499872+0000","last_clean":"2024-09-13T04:57:56.499872+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:57:56.499872+0000","last_undegraded":"2024-09-13T04:57:56.499872+0000","last_fullsized":"2024-09-13T04:57:56.499872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.161458+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:57:56.161458+0000","last_peered":"2024-09-13T04:57:56.161458+0000","last_clean":"2024-09-13T04:57:56.161458+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:57:56.161458+0000","last_undegraded":"2024-09-13T04:57:56.161458+0000","last_fullsized":"2024-09-13T04:57:56.161458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.500205+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:57:56.500205+0000","last_peered":"2024-09-13T04:57:56.500205+0000","last_clean":"2024-09-13T04:57:56.500205+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:57:56.500205+0000","last_undegraded":"2024-09-13T04:57:56.500205+0000","last_fullsized":"2024-09-13T04:57:56.500205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.154502+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:57:56.154502+0000","last_peered":"2024-09-13T04:57:56.154502+0000","last_clean":"2024-09-13T04:57:56.154502+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:57:56.154502+0000","last_undegraded":"2024-09-13T04:57:56.154502+0000","last_fullsized":"2024-09-13T04:57:56.154502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155387+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:57:56.155387+0000","last_peered":"2024-09-13T04:57:56.155387+0000","last_clean":"2024-09-13T04:57:56.155387+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:57:56.155387+0000","last_undegraded":"2024-09-13T04:57:56.155387+0000","last_fullsized":"2024-09-13T04:57:56.155387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T04:57:55.894891+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:57:55.894891+0000","last_peered":"2024-09-13T04:57:55.894891+0000","last_clean":"2024-09-13T04:57:55.894891+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:57:55.894891+0000","last_undegraded":"2024-09-13T04:57:55.894891+0000","last_fullsized":"2024-09-13T04:57:55.894891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":78884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":237568,"data_stored":78884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":290816,"data_stored":669233,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":290816,"data_stored":669233,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:58:01.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:58:01.566 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:01.567 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:58:01.584 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T04:58:00.909900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238616,"kb_used_data":1944,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754736104,"statfs":{"total":773094113280,"available":772849770496,"internally_reserved":0,"allocated":1990656,"data_stored":1811694,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.764613"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155829+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:57:56.155829+0000","last_peered":"2024-09-13T04:57:56.155829+0000","last_clean":"2024-09-13T04:57:56.155829+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:57:56.155829+0000","last_undegraded":"2024-09-13T04:57:56.155829+0000","last_fullsized":"2024-09-13T04:57:56.155829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.154929+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:57:56.154929+0000","last_peered":"2024-09-13T04:57:56.154929+0000","last_clean":"2024-09-13T04:57:56.154929+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:57:56.154929+0000","last_undegraded":"2024-09-13T04:57:56.154929+0000","last_fullsized":"2024-09-13T04:57:56.154929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155102+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:57:56.155102+0000","last_peered":"2024-09-13T04:57:56.155102+0000","last_clean":"2024-09-13T04:57:56.155102+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:57:56.155102+0000","last_undegraded":"2024-09-13T04:57:56.155102+0000","last_fullsized":"2024-09-13T04:57:56.155102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.499872+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:57:56.499872+0000","last_peered":"2024-09-13T04:57:56.499872+0000","last_clean":"2024-09-13T04:57:56.499872+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:57:56.499872+0000","last_undegraded":"2024-09-13T04:57:56.499872+0000","last_fullsized":"2024-09-13T04:57:56.499872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.161458+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:57:56.161458+0000","last_peered":"2024-09-13T04:57:56.161458+0000","last_clean":"2024-09-13T04:57:56.161458+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:57:56.161458+0000","last_undegraded":"2024-09-13T04:57:56.161458+0000","last_fullsized":"2024-09-13T04:57:56.161458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.500205+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:57:56.500205+0000","last_peered":"2024-09-13T04:57:56.500205+0000","last_clean":"2024-09-13T04:57:56.500205+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:57:56.500205+0000","last_undegraded":"2024-09-13T04:57:56.500205+0000","last_fullsized":"2024-09-13T04:57:56.500205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.154502+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:57:56.154502+0000","last_peered":"2024-09-13T04:57:56.154502+0000","last_clean":"2024-09-13T04:57:56.154502+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:57:56.154502+0000","last_undegraded":"2024-09-13T04:57:56.154502+0000","last_fullsized":"2024-09-13T04:57:56.154502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T04:57:56.155387+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:57:56.155387+0000","last_peered":"2024-09-13T04:57:56.155387+0000","last_clean":"2024-09-13T04:57:56.155387+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:57:56.155387+0000","last_undegraded":"2024-09-13T04:57:56.155387+0000","last_fullsized":"2024-09-13T04:57:56.155387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T04:57:55.894891+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:57:55.894891+0000","last_peered":"2024-09-13T04:57:55.894891+0000","last_clean":"2024-09-13T04:57:55.894891+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:57:55.894891+0000","last_undegraded":"2024-09-13T04:57:55.894891+0000","last_fullsized":"2024-09-13T04:57:55.894891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":78884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":237568,"data_stored":78884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":290816,"data_stored":669233,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":233472,"data_stored":78865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":290816,"data_stored":669233,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:58:01.585 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T04:58:01.585 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T04:58:01.869 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 23 2024-09-13T04:58:01.869 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 8850da77-4384-4335-9559-af2e805c4f72 2024-09-13T04:58:01.869 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-13T04:57:34.164121+0000 2024-09-13T04:58:01.869 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-09-13T04:57:56.147154+0000 2024-09-13T04:58:01.869 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T04:58:01.869 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 5 2024-09-13T04:58:01.869 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2024-09-13T04:58:01.869 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client jewel 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release reef 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 23 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout: removed_snaps_queue [2~1] 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 8 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/2346968854,v1:172.21.15.1:6820/2346968854] [v2:172.21.15.1:6822/2346968854,v1:172.21.15.1:6824/2346968854] exists,up 2fda29ca-3c6d-47ae-9244-8ccdf55f89f6 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 15 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6816/466532899,v1:172.21.15.1:6817/466532899] [v2:172.21.15.1:6819/466532899,v1:172.21.15.1:6821/466532899] exists,up 9ca6e867-faf0-4295-b8eb-4c738dd26cb9 2024-09-13T04:58:01.870 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6800/343712578,v1:172.21.15.1:6801/343712578] [v2:172.21.15.1:6802/343712578,v1:172.21.15.1:6803/343712578] exists,up d8159400-8662-4567-8714-b48c2d8f3ad2 2024-09-13T04:58:01.871 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6808/2021827348,v1:172.21.15.1:6809/2021827348] [v2:172.21.15.1:6810/2021827348,v1:172.21.15.1:6811/2021827348] exists,up 13a3ea70-fa37-41d8-bae7-0203952b02cb 2024-09-13T04:58:01.871 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/2183053562,v1:172.21.15.144:6801/2183053562] [v2:172.21.15.144:6802/2183053562,v1:172.21.15.144:6803/2183053562] exists,up e896e6b5-c54a-405f-ab69-2a5b7b7a153e 2024-09-13T04:58:01.871 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/2765053006,v1:172.21.15.144:6817/2765053006] [v2:172.21.15.144:6819/2765053006,v1:172.21.15.144:6821/2765053006] exists,up 7de23072-7eda-43d6-9b92-d608ef82c6fa 2024-09-13T04:58:01.871 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6818/2721913793,v1:172.21.15.144:6820/2721913793] [v2:172.21.15.144:6823/2721913793,v1:172.21.15.144:6825/2721913793] exists,up 8719fad9-81a0-46be-a546-db83bcace33b 2024-09-13T04:58:01.871 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/4202513770,v1:172.21.15.144:6809/4202513770] [v2:172.21.15.144:6810/4202513770,v1:172.21.15.144:6811/4202513770] exists,up 8bf0a351-e58e-4393-a6ce-fe70acf362aa 2024-09-13T04:58:01.888 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/2346968854,v1:172.21.15.1:6820/2346968854] [v2:172.21.15.1:6822/2346968854,v1:172.21.15.1:6824/2346968854] exists,up 2fda29ca-3c6d-47ae-9244-8ccdf55f89f6', 'osd.1 up in weight 1 up_from 15 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6816/466532899,v1:172.21.15.1:6817/466532899] [v2:172.21.15.1:6819/466532899,v1:172.21.15.1:6821/466532899] exists,up 9ca6e867-faf0-4295-b8eb-4c738dd26cb9', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6800/343712578,v1:172.21.15.1:6801/343712578] [v2:172.21.15.1:6802/343712578,v1:172.21.15.1:6803/343712578] exists,up d8159400-8662-4567-8714-b48c2d8f3ad2', 'osd.3 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6808/2021827348,v1:172.21.15.1:6809/2021827348] [v2:172.21.15.1:6810/2021827348,v1:172.21.15.1:6811/2021827348] exists,up 13a3ea70-fa37-41d8-bae7-0203952b02cb', 'osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/2183053562,v1:172.21.15.144:6801/2183053562] [v2:172.21.15.144:6802/2183053562,v1:172.21.15.144:6803/2183053562] exists,up e896e6b5-c54a-405f-ab69-2a5b7b7a153e', 'osd.5 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/2765053006,v1:172.21.15.144:6817/2765053006] [v2:172.21.15.144:6819/2765053006,v1:172.21.15.144:6821/2765053006] exists,up 7de23072-7eda-43d6-9b92-d608ef82c6fa', 'osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6818/2721913793,v1:172.21.15.144:6820/2721913793] [v2:172.21.15.144:6823/2721913793,v1:172.21.15.144:6825/2721913793] exists,up 8719fad9-81a0-46be-a546-db83bcace33b', 'osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/4202513770,v1:172.21.15.144:6809/4202513770] [v2:172.21.15.144:6810/4202513770,v1:172.21.15.144:6811/4202513770] exists,up 8bf0a351-e58e-4393-a6ce-fe70acf362aa'] 2024-09-13T04:58:01.888 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T04:58:02.006 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mon.a", 2024-09-13T04:58:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster": "ceph", 2024-09-13T04:58:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T04:58:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "admin_socket_mode": "", 2024-09-13T04:58:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T04:58:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_client_required": "cephx, none", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_cluster_required": "cephx", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_debug": "true", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_service_required": "cephx", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_supported": "cephx", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio": "true", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_reap_max": "16", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_async_discard": "false", 2024-09-13T04:58:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_block_size": "4096", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio": "false", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_enable_discard": "false", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_flock_retry": "3", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_inject_crash": "0", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring": "false", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring_hipri": "false", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T04:58:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_type": "", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_allocator": "hybrid", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_buffered_io": "true", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T04:58:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_replay_recovery": "false", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_sync_write": "false", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T04:58:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_allocator": "hybrid", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_create": "true", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_create": "false", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_path": "", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_size": "0", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_path": "", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T04:58:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_size": "107374182400", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_create": "false", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_path": "", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs": "true", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T04:58:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_autotune": "true", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size": "0", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_type": "2q", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_clone_cow": "true", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T04:58:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_mode": "none", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_freelist": "false", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-13T04:58:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T04:58:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fail_eio": "false", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T04:58:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T04:58:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size": "0", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T04:58:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T04:58:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_mem": "512", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T04:58:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_tracing": "false", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T04:58:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_cluster_require_version": "2", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_require_signatures": "false", 2024-09-13T04:58:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_require_version": "2", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_service_require_signatures": "false", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_service_require_version": "2", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_sign_messages": "true", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "chdir": "", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_acl_type": "", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_asio_thread_count": "2", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_cache_mid": "0.750000", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_cache_size": "16384", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_caps_release_delay": "5", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_check_pool_perm": "true", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_force_sync_read": "false", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_getattr_caps": "false", 2024-09-13T04:58:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_inject_features": "", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_die_on_failed_remount": "false", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_dirsize_rbytes": "true", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_force_lazyio": "false", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_fs": "", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_inject_release_failure": "false", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_max_inline_size": "4096", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mds_namespace": "", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_metadata": "", 2024-09-13T04:58:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_gid": "-1", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_timeout": "300", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_uid": "-1", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mountpoint": "/", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_notify_timeout": "10", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc": "true", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_objects": "1000", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_size": "209715200", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_permissions": "true", 2024-09-13T04:58:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_quota": "true", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_quota_df": "true", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_max_bytes": "0", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_max_periods": "4", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_min": "131072", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_reconnect_stale": "false", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_shutdown_timeout": "30", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_snapdir": ".snap", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_tick_interval": "1", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_trace": "", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_use_faked_inos": "false", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_use_random_mds": "false", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog": "false", 2024-09-13T04:58:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog_port": "12201", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_monitors": "default=true", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog": "false", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog_level": "info", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "-", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_network": "", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_network_interface": "", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_isal": "false", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_level": "5", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zstd_level": "1", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T04:58:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_seastar_smp": "1", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location_hook": "", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location_hook_timeout": "10", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "daemonize": "false", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "daos_pool": "tank", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T04:58:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_alienstore": "0/5", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asok": "1/5", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asok_assert_abort": "false", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_auth": "20/20", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bdev": "1/3", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bluefs": "1/5", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bluestore": "1/5", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_buffer": "0/1", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_cephsqlite": "0/5", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_client": "0/5", 2024-09-13T04:58:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_compressor": "1/5", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_context": "0/1", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_crush": "1/1", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_crypto": "1/5", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_cyanstore": "0/5", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_dpdk": "1/5", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_eventtrace": "1/5", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_filer": "0/1", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_filestore": "1/3", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_finisher": "1/1", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_fuse": "1/5", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T04:58:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_javaclient": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_journal": "1/3", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_journaler": "0/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_kstore": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_leveldb": "4/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_lockdep": "0/1", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mclock": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_balancer": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_locker": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_log": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T04:58:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_migrator": "1/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_memstore": "1/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mgr": "2/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mgrc": "1/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mon": "20/20", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_monc": "0/10", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_ms": "1/1", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_none": "0/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objclass": "0/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objectcacher": "0/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objecter": "0/1", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_optracker": "0/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_osd": "1/5", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_paxos": "20/20", 2024-09-13T04:58:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_perfcounter": "1/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_prioritycache": "1/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rados": "0/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd": "0/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_replay": "0/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_refs": "0/0", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_reserver": "1/1", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw": "1/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_access": "1/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_flight": "1/5", 2024-09-13T04:58:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_sync": "1/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rocksdb": "4/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_backref": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_cache": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_device": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_epm": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_journal": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_lba": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_odata": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_omap": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_onode": "0/5", 2024-09-13T04:58:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_t": "0/5", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_seastore_tm": "0/5", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_striper": "0/1", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_test": "0/5", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_throttle": "1/1", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_timer": "0/1", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_tp": "0/5", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "device_failure_prediction_mode": "none", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_graylog": "false", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_journald": "false", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_stderr": "true", 2024-09-13T04:58:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_syslog": "false", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event_tracing": "false", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_http_port": "9926", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_prio_limit": "5", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_sort_metrics": "true", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "exporter_stats_period": "5", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "fake_statfs_for_testing": "0", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "fatal_signal_handlers": "true", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "filer_max_purge_ops": "10", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "filer_max_truncate_ops": "128", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T04:58:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_blackhole": "false", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_btrfs_snap": "true", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_caller_concurrency": "10", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_omap_check": "false", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_verify_split": "false", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_dump_file": "", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T04:58:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fadvise": "true", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fail_eio": "true", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fd_cache_size": "128", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fiemap": "false", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_inject_stall": "0", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_parallel": "false", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_trailing": "false", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_writeahead": "false", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_kill_at": "0", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T04:58:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_merge_threshold": "-10", 2024-09-13T04:58:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_odsync_write": "false", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_backend_path": "", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_threads": "2", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_punch_hole": "false", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T04:58:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_ops": "50", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_seek_data_hole": "false", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_sloppy_crc": "false", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_splice": "false", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_split_multiple": "2", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_split_rand_factor": "20", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_update_to": "1000", 2024-09-13T04:58:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T04:58:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_xfs_extsize": "false", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_zfs_snap": "false", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fio_dir": "/tmp/fio", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "8850da77-4384-4335-9559-af2e805c4f72", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_allow_other": "true", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_big_writes": "true", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_debug": "false", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_default_permissions": "false", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_disable_pagecache": "false", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_max_write": "0", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_multithreaded": "true", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_require_active_mds": "true", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_set_user_groups": "true", 2024-09-13T04:58:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_move": "true", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_read": "true", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_write": "true", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "gss_target_name": "ceph", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_file": "", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_inject_failure": "0", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_interval": "5", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "host": "smithi001", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T04:58:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "inject_early_sigterm": "false", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "jaeger_agent_port": "6799", 2024-09-13T04:58:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "jaeger_tracing_enable": "false", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_aio": "true", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_align_min_size": "65536", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_block_align": "true", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_block_size": "4096", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_dio": "true", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_discard": "false", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_force_aio": "false", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_ignore_corruption": "false", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_max_write_entries": "100", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_replay_from": "0", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T04:58:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_write_header_frequency": "0", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_zero_on_create": "false", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_prefetch_periods": "10", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_prezero_periods": "5", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_write_head_interval": "15", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "keyfile": "", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_backend": "rocksdb", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T04:58:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_max_bytes": "67108864", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_max_ops": "512", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_onode_map_size": "1024", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_sync_transaction": "false", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_block_size": "65536", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_bloom_size": "0", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_cache_size": "536870912", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_compact_on_mount": "false", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_compression": "false", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_log": "", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-13T04:58:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_max_open_files": "0", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_paranoid": "false", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "librados_thread_count": "2", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "lockdep": "false", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "lockdep_force_backtrace": "false", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_coarse_timestamps": "true", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_flush_on_exit": "false", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_graylog_port": "12201", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_max_new": "1000", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_max_recent": "10000", 2024-09-13T04:58:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_stderr_prefix": "", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": "true", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_graylog": "false", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_journald": "false", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_stderr": "false", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_syslog": "false", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_action_on_write_error": "1", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_alternate_name_max": "8192", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_asio_thread_count": "2", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_export_pin": "true", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T04:58:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_interval": "10", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_max": "-1", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_max_until": "-1", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_merge_size": "50", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_mode": "0", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T04:58:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_bits": "3", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_size": "10000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T04:58:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_mid": "0.700000", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_interval": "1", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T04:58:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_auth_pins": "false", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_frag": "false", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_scatterstat": "false", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_subtrees": "false", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_default_dir_hash": "2", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_defer_session_stale": "true", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_max_entries": "0", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_prefetch": "true", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T04:58:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_early_reply": "true", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_enable_op_tracker": "true", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_enforce_unique_name": "true", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T04:58:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_health_dummy": "false", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T04:58:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_join_fs": "", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_journal_format": "1", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_create_at": "0", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_export_at": "0", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_import_at": "0", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_at": "0", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_link_at": "0", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T04:58:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_openc_at": "0", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_rename_at": "0", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_max_events": "-1", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_max_segments": "128", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_pause": "false", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_segment_size": "0", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_completed_requests": "100000", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_export_size": "20971520", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_file_recover": "32", 2024-09-13T04:58:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_files": "64", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_ops": "8192", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_metrics_update_interval": "2", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_min_caps_per_client": "100", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_numa_node": "-1", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_duration": "600", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_size": "20", 2024-09-13T04:58:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_log_threshold": "5", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_ping_grace": "15", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_ping_interval": "5", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_caps": "30000", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_replay_interval": "1.000000", 2024-09-13T04:58:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_root_ino_gid": "0", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_root_ino_uid": "0", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T04:58:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_shutdown_check": "0", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_skip_ino": "0", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_min_uid": "0", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_rstat": "false", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_symlink_recovery": "true", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_thrash_exports": "0", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_thrash_fragments": "0", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_tick_interval": "5.000000", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_valgrind_exit": "false", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_verify_backtrace": "1", 2024-09-13T04:58:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_verify_scatter": "false", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_wipe_sessions": "false", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool_debug": "false", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_page_set": "false", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_page_size": "65536", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_bytes": "134217728", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_messages": "512", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T04:58:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_disabled_modules": "", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mds_messages": "128", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mon_messages": "128", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_osd_messages": "8192", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_pool": "true", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T04:58:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_standby_modules": "true", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_stats_period": "5", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_stats_threshold": "5", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_tick_period": "2", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_allow_pool_delete": "true", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_bytes": "104857600", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T04:58:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_file_level": "debug", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T04:58:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_start": "false", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_trim": "true", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T04:58:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cpu_threads": "4", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_avail_crit": "5", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_avail_warn": "5", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_json": "false", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_extra_checks": "false", 2024-09-13T04:58:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_quincy": "false", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_election_default_strategy": "1", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_election_timeout": "5.000000", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T04:58:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_enable_op_tracker": "true", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_fake_pool_delete": "false", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_force_quorum_join": "false", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_log_update_period": "5", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_max_detail": "50", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog": "true", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_host": "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.144", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_host_override": "", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_initial_members": "", 2024-09-13T04:58:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease": "5.000000", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_log_full_interval": "50", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_log_max": "10000", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_log_max_summary": "50", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_log_epochs": "500", 2024-09-13T04:58:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_osd": "10000", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_memory_autotune": "true", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_memory_target": "2147483648", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T04:58:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_digest_period": "5", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_min_osdmap_epochs": "500", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_complaint_time": "30", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_duration": "600", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_size": "20", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_log_threshold": "5", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T04:58:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_cache_size": "500", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T04:58:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T04:58:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_report_timeout": "900", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T04:58:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_max_osds": "4", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T04:58:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_interval": "86400", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_max_keys": "100", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_timeout": "300", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_session_timeout": "300", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_smart_report_timeout": "5", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_pool_size": "4", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_debug": "false", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T04:58:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_tick_interval": "5", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T04:58:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_older_version": "true", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T04:58:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": "", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "motr_tracing_enabled": "false", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_op_threads": "3", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_cm": "false", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_device_name": "", 2024-09-13T04:58:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_sl": "3", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_type": "ib", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_reap_threshold": "5", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_before_connect": "false", 2024-09-13T04:58:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_ipv4": "true", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_ipv6": "false", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_msgr1": "true", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_msgr2": "true", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_port_max": "7568", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_port_min": "6800", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_retry_count": "3", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_retry_delay": "5", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_client": "false", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mds": "false", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mgr": "false", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mon": "false", 2024-09-13T04:58:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_osd": "false", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_client_mode": "crc secure", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_cluster_type": "", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_compress_secure": "false", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_crc_data": "true", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_crc_header": "true", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_bug": "true", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_old_message": "true", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T04:58:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hugepages": "", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_lro": "true", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T04:58:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_pmd": "", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_port_id": "0", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dump_on_send": "false", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_type": "", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_network_congestion": "0", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T04:58:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_max_accept_failures": "4", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_max_backoff": "15.000000", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_osd_compress_mode": "none", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_pq_min_cost": "65536", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_public_type": "", 2024-09-13T04:58:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_service_mode": "crc secure", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_nodelay": "true", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_type": "async+posix", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "no_config_file": "false", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inflight_ops": "1024", 2024-09-13T04:58:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_timeout": "10.000000", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "objectstore_blackhole": "false", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "openssl_engine_opts": "", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_max_ops": "4", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_slop": "0.020000", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T04:58:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_asio_thread_count": "2", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_scan_max": "512", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_scan_min": "64", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_peering": "false", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_beacon_report_interval": "300", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_duration": "30", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T04:58:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_blkin_trace_all": "false", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_update_on_start": "true", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_message_cap": "256", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T04:58:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_op_priority": "63", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_watch_timeout": "30", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_command_max_records": "256", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_compact_on_start": "false", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_update_on_start": "true", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T04:58:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_op_order": "false", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_shutdown": "false", 2024-09-13T04:58:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T04:58:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_default_notify_timeout": "30", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_enable_op_tracker": "true", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_info": "true", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_shutdown": "true", 2024-09-13T04:58:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_function_tracing": "false", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_grace": "20", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_interval": "6", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_stale": "600", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T04:58:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal_size": "5120", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_kill_backfill_at": "0", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_cache_size": "50", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_dedup": "true", 2024-09-13T04:58:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_message_max": "40", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_attr_name_len": "100", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_attr_size": "0", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_backfills": "1", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_markdown_count": "5", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_markdown_period": "600", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_name_len": "2048", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_size": "134217728", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T04:58:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pgls": "1024", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_push_cost": "8388608", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_push_objects": "10", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_scrubs": "3", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_write_size": "90", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T04:58:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_profile": "balanced", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T04:58:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_base": "805306368", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_target": "4294967296", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_target_autotune": "false", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_min_recovery_priority": "0", 2024-09-13T04:58:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_report_interval": "5", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_num_cache_shards": "32", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_node": "-1", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objecter_finishers": "1", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore": "bluestore", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore_fuse": "false", 2024-09-13T04:58:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore_tracing": "false", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_duration": "600", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_size": "20", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_log_threshold": "5", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards": "0", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T04:58:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_thread_timeout": "15", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_open_classes_on_start": "true", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_os_flags": "0", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_peering_op_priority": "255", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_delete_priority": "5", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T04:58:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_crimson": "false", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T04:58:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flags": "0", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_min_size": "0", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_size": "2", 2024-09-13T04:58:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_type": "replicated", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_cost": "20971520", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active": "0", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T04:58:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_op_priority": "3", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_priority": "5", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_repair_during_recovery": "false", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T04:58:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_cost": "52428800", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_end_hour": "0", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T04:58:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_priority": "5", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_skip_data_digest": "false", 2024-09-13T04:58:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_smart_report_timeout": "5", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_priority": "5", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_target_transaction_size": "30", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T04:58:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tracing": "false", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_use_stale_snap": "false", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_kill_at": "0", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_max_join_drift": "10", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_min": "500", 2024-09-13T04:58:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_min_wait": "0.050000", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_service_trim_max": "500", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_service_trim_min": "250", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_stash_full_interval": "25", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_trim_max": "500", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_trim_min": "250", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "perf": "true", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "-", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrv": "", 2024-09-13T04:58:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "public_bind_addr": "-", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "public_network": "", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "public_network_interface": "", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "qat_compressor_enabled": "false", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_mon_op_timeout": "0", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_osd_op_timeout": "0", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_tracing": "false", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_atime_update_interval": "60", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T04:58:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache": "true", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_policy": "writearound", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_size": "33554432", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_compression_hint": "none", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T04:58:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_clone_format": "auto", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_data_pool": "", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_features": "61", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_format": "2", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_map_options": "", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_order": "22", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_pool": "rbd", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_stripe_count": "0", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T04:58:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_io_scheduler": "simple", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T04:58:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_order": "24", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_pool": "", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_splay_width": "4", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T04:58:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T04:58:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T04:58:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_op_threads": "1", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_plugins": "", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T04:58:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T04:58:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_tracing": "false", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_validate_names": "true", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_validate_pool": "true", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "restapi_base_url": "", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "restapi_log_level": "", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T04:58:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_admin_entry": "admin", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_backend_store": "rados", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_barbican_url": "", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_beast_enable_async": "true", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_enabled": "true", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T04:58:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_lru_size": "10000", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_config_store": "rados", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_content_length_compat": "false", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cross_domain_policy": "", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T04:58:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T04:58:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T04:58:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T04:58:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_window": "30", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T04:58:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T04:58:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dns_name": "", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dns_s3website_name": "", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_ops_log": "false", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_static_website": "false", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_usage_log": "false", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_expose_bucket": "false", 2024-09-13T04:58:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_extended_http_attrs": "", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_filter": "none", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_objs": "32", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T04:58:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_init_timeout": "300", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_password": "", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_project": "", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T04:58:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_token": "", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_user": "", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_api_version": "2", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T04:58:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_url": "", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_objs": "32", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_worker": "3", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T04:58:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_http_headers": "", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attr_size": "0", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T04:58:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_listing_results": "1000", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_notify_retries": "10", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T04:58:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T04:58:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_num_control_oids": "8", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_numa_node": "-1", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T04:58:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_tracing": "false", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_token": "", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_url": "", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_rados": "false", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T04:58:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_print_continue": "true", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_tracing": "false", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_realm": "", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_realm_id": "", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_region": "", 2024-09-13T04:58:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_request_uri": "", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_resolve_cname": "false", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_run_sync_thread": "true", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T04:58:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_script_uri": "", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_service_provider_name": "", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_client_id": "", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_client_secret": "", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_entry": "sts", 2024-09-13T04:58:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_key": "sts", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_auth_url": "", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_custom_header": "", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_need_stats": "true", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_tenant_name": "", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_url": "", 2024-09-13T04:58:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_lease_period": "120", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T04:58:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_thread_pool_size": "512", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_comment": "", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_createby": "", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_encoding": "", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_flag": "false", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_origin": "", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_tracker": "", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_max_shards": "32", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T04:58:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_use_opa_authz": "false", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_unique_email": "true", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_verify_ssl": "true", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zone": "", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zone_id": "", 2024-09-13T04:58:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zonegroup": "", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zonegroup_id": "", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_block_size": "4096", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T04:58:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_partition_filters": "false", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_perf": "false", 2024-09-13T04:58:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "run_dir": "/var/run/ceph", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_block_create": "true", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_device_size": "53687091200", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T04:58:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_main_device_type": "SSD", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "seastore_segment_size": "67108864", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "set_keepcaps": "false", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "setgroup": "", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "setuser": "", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "setuser_match_path": "", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "thp": "false", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "threadpool_default_timeout": "60", 2024-09-13T04:58:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T04:58:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "throttler_perf_counter": "true" 2024-09-13T04:58:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:02.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T04:58:02.295 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a: {} 2024-09-13T04:58:02.296 INFO:teuthology.orchestra.run.smithi001.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T04:58:02.309 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b: {} 2024-09-13T04:58:02.309 INFO:teuthology.orchestra.run.smithi001.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T04:58:02.318 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c: {} 2024-09-13T04:58:02.318 INFO:teuthology.orchestra.run.smithi001.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T04:58:02.334 DEBUG:teuthology.orchestra.run.smithi001:> type ceph-objectstore-tool 2024-09-13T04:58:02.338 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T04:58:02.338 DEBUG:teuthology.orchestra.run.smithi144:> type ceph-objectstore-tool 2024-09-13T04:58:02.344 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T04:58:02.344 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-13T04:58:02.356 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-13T04:58:02.356 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T04:58:02.356 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:58:02.356 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:58:02.357 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-13T04:58:02.357 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T04:58:02.357 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T04:58:02.357 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-13T04:58:02.357 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T04:58:02.358 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T04:58:02.358 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:02.358 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T04:58:02.358 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:02.358 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T04:58:02.359 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:58:02.359 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-13T04:58:02.388 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:02.388+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:02.401 INFO:teuthology.orchestra.run.smithi001.stdout:mapped writes DISABLED 2024-09-13T04:58:02.401 INFO:teuthology.orchestra.run.smithi001.stdout:mapped reads DISABLED 2024-09-13T04:58:02.401 INFO:teuthology.orchestra.run.smithi001.stdout:Seed set to 393 2024-09-13T04:58:02.460 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:02.460+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:02.503 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:02.503 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:02.517 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:02.517 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T04:58:02.517 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:02.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:02.523 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:02.523 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:02.541 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:02.558 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:02.558 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:02.571 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:02.572+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:02.576 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:02.576 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:02.595 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:02.596 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:02.612 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:02.613 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:02.632 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:02.632 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:02.634 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:02.634 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:02.647 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:02.651 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:02.661 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:02.660+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:02.762 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:02.760+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:02.763 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:02.763 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:02.779 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:02.865 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:02.864+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:02.945 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:02.945 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:02.958 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:03.001 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:03.000+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:03.065 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:03.064+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:03.115 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:03.115 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:03.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:03.173 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:03.172+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:58:03.265 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:03.265 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:03.267 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:03.264+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:03.279 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:03.369 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:03.368+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:03.446 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:03.446 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:03.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:03.505 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:03.504+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:03.570 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:03.568+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:03.633 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:03.633 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:03.647 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:03.672 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:03.672+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:03.773 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:03.772+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:03.817 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:03.817 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:03.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:03.874 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:03.872+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:03.974 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:03.972+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:04.014 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:04.015 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:04.029 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:04.075 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:04.072+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:04.175 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:04.176+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:04.200 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:04.201 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:04.216 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:04.276 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:04.272+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:04.340 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:04.340 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:04.355 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:04.404 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:04.404+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:04.477 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:04.476+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:04.485 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:04.485 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:04.499 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:04.578 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:04.576+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:04.668 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:04.668 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:04.668 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:04.682 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:04.713 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:04.712+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:04.779 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:04.780+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:04.805 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:04.805 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:04.819 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:04.825 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:04.825 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:04.841 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:04.857 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:04.858 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:04.873 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:04.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:04.904 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:04.904+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:04.913 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:04.931 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:04.931 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:04.949 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:04.949 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:04.981 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:04.980+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:04.985 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:04.985 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:04.985 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:04.985 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:04.999 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:05.082 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:05.080+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:05.159 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:05.159 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:05.159 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:05.173 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:05.183 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:05.180+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:05.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:58:05.283 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:05.284+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:05.296 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:67 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'41, mlcod=18'39)", 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.254596+0000", 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.040452861, 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.007278657, 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.254596+0000", 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.254596+0000", 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.254594+0000", 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.254609+0000", 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.469e-05 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.254637+0000", 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7555999999999998e-05 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.254649+0000", 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2214e-05 2024-09-13T04:58:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.254768+0000", 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011869199999999999 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.254825+0000", 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6866000000000001e-05 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.261791+0000", 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069666709999999998 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.261875+0000", 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3712000000000001e-05 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:68 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'42, mlcod=18'39)", 2024-09-13T04:58:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.255539+0000", 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.039509880000001, 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.006396629, 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.255539+0000", 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.255539+0000", 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.255537+0000", 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.255551+0000", 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3376e-05 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.255579+0000", 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8396999999999999e-05 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.255592+0000", 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2622999999999999e-05 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.258690+0000", 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0030984530000000001 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.258742+0000", 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2033999999999997e-05 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.261880+0000", 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0031376730000000001 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.261936+0000", 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5875999999999998e-05 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.305 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:69 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'43, mlcod=18'39)", 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.256863+0000", 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.038186581, 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0051375029999999999, 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.256863+0000", 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.256863+0000", 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.256861+0000", 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.256916+0000", 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5522999999999997e-05 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.256944+0000", 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7886e-05 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.256956+0000", 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2284e-05 2024-09-13T04:58:05.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.259505+0000", 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002548261 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.259540+0000", 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5509999999999997e-05 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.261942+0000", 2024-09-13T04:58:05.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002401519 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.262000+0000", 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8337000000000001e-05 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:89 1.0 e18/17 1:30306672:devicehealth::main.db.0000000000000000:head v 18'59, mlcod=18'57)", 2024-09-13T04:58:05.310 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.298449+0000", 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.996599997000001, 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039519530000000002, 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.298449+0000", 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.298449+0000", 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.298447+0000", 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.298506+0000", 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8773000000000001e-05 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.298535+0000", 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9213e-05 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.298549+0000", 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3896999999999999e-05 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.298644+0000", 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4776000000000002e-05 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.298696+0000", 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2259000000000002e-05 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.302347+0000", 2024-09-13T04:58:05.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036510090000000002 2024-09-13T04:58:05.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.302401+0000", 2024-09-13T04:58:05.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3885999999999999e-05 2024-09-13T04:58:05.315 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.315 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:108 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'70, mlcod=18'64)", 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.446626+0000", 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.848423429, 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0042837800000000001, 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.446626+0000", 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.446626+0000", 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.446624+0000", 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.446639+0000", 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5403000000000001e-05 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.446669+0000", 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9842999999999999e-05 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.446694+0000", 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5145999999999999e-05 2024-09-13T04:58:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.447398+0000", 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00070382099999999998 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.447428+0000", 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9790000000000001e-05 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450853+0000", 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034246699999999999 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450909+0000", 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6820000000000001e-05 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.320 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:109 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'71, mlcod=18'64)", 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.447101+0000", 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.84794791, 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0085761569999999992, 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.447101+0000", 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.447101+0000", 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.447099+0000", 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.447172+0000", 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2620000000000006e-05 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.447199+0000", 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6893000000000002e-05 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.447215+0000", 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6325999999999999e-05 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448007+0000", 2024-09-13T04:58:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00079170399999999995 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448043+0000", 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6418999999999998e-05 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455622+0000", 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0075785690000000003 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455677+0000", 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5646000000000002e-05 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:110 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'72, mlcod=18'66)", 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.448276+0000", 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.846773153000001, 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0074363700000000003, 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.325 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448276+0000", 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448276+0000", 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448274+0000", 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448287+0000", 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3039e-05 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448310+0000", 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3044000000000002e-05 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448330+0000", 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.946e-05 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.449671+0000", 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0013406010000000001 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.449703+0000", 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2617000000000003e-05 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455683+0000", 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059802750000000002 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455712+0000", 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8884e-05 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.329 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:111 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'73, mlcod=18'66)", 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.448726+0000", 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.846322957, 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0070114269999999998, 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448726+0000", 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448726+0000", 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448725+0000", 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448737+0000", 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2237e-05 2024-09-13T04:58:05.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448764+0000", 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6795999999999999e-05 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.448779+0000", 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4583000000000001e-05 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450267+0000", 2024-09-13T04:58:05.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014883909999999999 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450308+0000", 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0979999999999997e-05 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455714+0000", 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0054061270000000002 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455738+0000", 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3560000000000001e-05 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:112 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'74, mlcod=18'67)", 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.449459+0000", 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.845590047, 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0063079429999999999, 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.334 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.449459+0000", 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.449459+0000", 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.449458+0000", 2024-09-13T04:58:05.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.449533+0000", 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5326000000000004e-05 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.449561+0000", 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8496e-05 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.449574+0000", 2024-09-13T04:58:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.253e-05 2024-09-13T04:58:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450989+0000", 2024-09-13T04:58:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014152780000000001 2024-09-13T04:58:05.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.451019+0000", 2024-09-13T04:58:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9556999999999998e-05 2024-09-13T04:58:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455739+0000", 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0047200300000000001 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455767+0000", 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8189000000000001e-05 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:113 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'75, mlcod=18'67)", 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.450162+0000", 2024-09-13T04:58:05.339 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.844887140000001, 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0056572189999999998, 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450162+0000", 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450162+0000", 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450161+0000", 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450174+0000", 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3247e-05 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450197+0000", 2024-09-13T04:58:05.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3056000000000001e-05 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450213+0000", 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5483999999999999e-05 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.452312+0000", 2024-09-13T04:58:05.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002099685 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.452344+0000", 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2279e-05 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455768+0000", 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034239050000000001 2024-09-13T04:58:05.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455819+0000", 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0785999999999998e-05 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:114 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'76, mlcod=18'68)", 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.450854+0000", 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.844195552, 2024-09-13T04:58:05.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0050064369999999999, 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450854+0000", 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450854+0000", 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450852+0000", 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450865+0000", 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2999999999999999e-05 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450883+0000", 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7453e-05 2024-09-13T04:58:05.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.450894+0000", 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1087000000000001e-05 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.452853+0000", 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019593729999999999 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.452884+0000", 2024-09-13T04:58:05.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1016000000000003e-05 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455822+0000", 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0029377549999999998 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455860+0000", 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8142999999999997e-05 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:115 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'77, mlcod=18'68)", 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.451330+0000", 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.843719442999999, 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045641140000000002, 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.451330+0000", 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.451330+0000", 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.451328+0000", 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.451441+0000", 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000112596 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.451503+0000", 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2513000000000003e-05 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.451531+0000", 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7826000000000001e-05 2024-09-13T04:58:05.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.453435+0000", 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0019041399999999999 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.453476+0000", 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0697e-05 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455863+0000", 2024-09-13T04:58:05.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023873129999999998 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.455894+0000", 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0309000000000001e-05 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:119 1.0 e18/17 1:30306672:devicehealth::main.db.0000000000000000:head v 18'81, mlcod=18'78)", 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.459922+0000", 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.835126589, 2024-09-13T04:58:05.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046271699999999999, 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.459922+0000", 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.459922+0000", 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.459921+0000", 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.459995+0000", 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3685999999999999e-05 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.460014+0000", 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9182999999999999e-05 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.460028+0000", 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3890000000000001e-05 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.460102+0000", 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3856000000000006e-05 2024-09-13T04:58:05.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.460183+0000", 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0890000000000006e-05 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.464496+0000", 2024-09-13T04:58:05.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043125459999999996 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.464550+0000", 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4135999999999998e-05 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:137 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'95, mlcod=18'91)", 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:50.951791+0000", 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.343258035, 2024-09-13T04:58:05.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045293979999999996, 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.951791+0000", 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.951791+0000", 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.951790+0000", 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.951886+0000", 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5617000000000003e-05 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.951899+0000", 2024-09-13T04:58:05.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.951905+0000", 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0020000000000003e-06 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.951941+0000", 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5352000000000003e-05 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.951960+0000", 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9462000000000001e-05 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.956286+0000", 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043258020000000001 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:50.956320+0000", 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4422999999999999e-05 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.362 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:153 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'107, mlcod=18'105)", 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:51.688629+0000", 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.606419979, 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005432966, 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.688629+0000", 2024-09-13T04:58:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.688629+0000", 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.688627+0000", 2024-09-13T04:58:05.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.688648+0000", 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0673e-05 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.688675+0000", 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7676999999999999e-05 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.688691+0000", 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5869999999999999e-05 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.688781+0000", 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0155999999999995e-05 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.688834+0000", 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2666e-05 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.693978+0000", 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0051438220000000002 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.694062+0000", 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4269000000000002e-05 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:154 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'108, mlcod=18'105)", 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:51.689194+0000", 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.605855084, 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049313880000000001, 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689194+0000", 2024-09-13T04:58:05.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689194+0000", 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689192+0000", 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689212+0000", 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0293999999999999e-05 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689245+0000", 2024-09-13T04:58:05.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2566e-05 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689261+0000", 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6147e-05 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689636+0000", 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00037529600000000001 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689683+0000", 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.655e-05 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.694070+0000", 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043867649999999999 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.694125+0000", 2024-09-13T04:58:05.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5664000000000001e-05 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:155 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'109, mlcod=18'105)", 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:51.689797+0000", 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.605251909, 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0069190529999999997, 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689797+0000", 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689797+0000", 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.689795+0000", 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.690023+0000", 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022708200000000001 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.690053+0000", 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0246000000000001e-05 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.690069+0000", 2024-09-13T04:58:05.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.607e-05 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.690431+0000", 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00036238699999999998 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.690488+0000", 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6616000000000001e-05 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.696637+0000", 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0061486240000000001 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.696716+0000", 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9721999999999993e-05 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:156 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'110, mlcod=18'105)", 2024-09-13T04:58:05.377 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:51.690984+0000", 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.604065193, 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0058019270000000001, 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.690984+0000", 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.690984+0000", 2024-09-13T04:58:05.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.690982+0000", 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691005+0000", 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2900000000000001e-05 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691029+0000", 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4406e-05 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691045+0000", 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5529999999999999e-05 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.692328+0000", 2024-09-13T04:58:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001282849 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.692374+0000", 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6292999999999999e-05 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.696720+0000", 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043459120000000004 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.696786+0000", 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5790000000000005e-05 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:157 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'111, mlcod=18'105)", 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:51.691654+0000", 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.603395445, 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0051953379999999999, 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.382 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691654+0000", 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691654+0000", 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691653+0000", 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691683+0000", 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0562999999999999e-05 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691702+0000", 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8839999999999999e-05 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.691716+0000", 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.376e-05 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.693171+0000", 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001455497 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.693231+0000", 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9889000000000002e-05 2024-09-13T04:58:05.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.696792+0000", 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035609729999999998 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.696849+0000", 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6675999999999997e-05 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4163.0:158 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'112, mlcod=18'105)", 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:51.692546+0000", 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.602503296, 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043611259999999999, 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.692546+0000", 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.692546+0000", 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.692544+0000", 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.692653+0000", 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108595 2024-09-13T04:58:05.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.692710+0000", 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6647000000000002e-05 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.692757+0000", 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6879000000000003e-05 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.694230+0000", 2024-09-13T04:58:05.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0014739079999999999 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.694277+0000", 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6919000000000003e-05 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.696851+0000", 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002574144 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:51.696907+0000", 2024-09-13T04:58:05.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5444e-05 2024-09-13T04:58:05.391 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.391 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.391 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.391 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.391 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.391 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:05.393 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:05.393 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:05.392+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:05.493 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:05.492+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:05.564 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:05.564 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:05.578 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:05.593 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:05.592+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:05.695 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:05.692+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:05.737 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:05.738 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:05.738 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:05.738 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:05.738 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:05.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:05.800 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:05.800+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4245.0:5 2.2 e22/20 2:5cea7035:::rbd_info:head v 22'1, mlcod=0'0)", 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:55.159081+0000", 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 10.717679391000001, 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002107023, 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.876 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159081+0000", 2024-09-13T04:58:05.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159081+0000", 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159081+0000", 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159089+0000", 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9099999999999994e-06 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159117+0000", 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7653000000000001e-05 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159142+0000", 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-05 2024-09-13T04:58:05.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159267+0000", 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000125143 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159349+0000", 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1961999999999999e-05 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.161124+0000", 2024-09-13T04:58:05.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001774905 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.161188+0000", 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4423000000000003e-05 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4245.0:7 2.2 e23/20 2:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:56.158816+0000", 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7179448659999998, 2024-09-13T04:58:05.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020319090000000001, 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158816+0000", 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158816+0000", 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158815+0000", 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158824+0000", 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9670000000000007e-06 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158852+0000", 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8183000000000001e-05 2024-09-13T04:58:05.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158867+0000", 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4316e-05 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158984+0000", 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000116989 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.159056+0000", 2024-09-13T04:58:05.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2862999999999997e-05 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.160779+0000", 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001722105 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.160848+0000", 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8992999999999995e-05 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.888 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:05.889 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:05.889 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:05.890 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:05.915 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:58:05.932 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:05.932+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:05.998 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:05.996+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:06.051 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:06.051 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:06.051 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:06.051 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:06.065 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:06.112 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:06.112+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:06.188 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:06.188 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:06.188 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:06.200 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:06.196+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:06.202 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:06.300 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:06.300+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:06.362 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:06.362 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:06.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:06.362 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:06.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.363 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:12 3.8 e27/24 3:173a9b69:::rbd_header.112a8b98d47:head v 27'1, mlcod=0'0)", 2024-09-13T04:58:06.363 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:05.931015+0000", 2024-09-13T04:58:06.363 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.43207873299999999, 2024-09-13T04:58:06.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001284758, 2024-09-13T04:58:06.363 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931015+0000", 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931015+0000", 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931015+0000", 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931019+0000", 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2239999999999997e-06 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931028+0000", 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1579999999999994e-06 2024-09-13T04:58:06.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931037+0000", 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0000000000000002e-06 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931100+0000", 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2998999999999998e-05 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931133+0000", 2024-09-13T04:58:06.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3203e-05 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.932263+0000", 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001129583 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.932300+0000", 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6912999999999997e-05 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:06.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:13 3.c e27/24 3:32c90b40:::rbd_object_map.112a8b98d47:head v 27'1, mlcod=0'0)", 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:05.933323+0000", 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.42977062700000002, 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0022159739999999999, 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933323+0000", 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933323+0000", 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933322+0000", 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933328+0000", 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8649999999999998e-06 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933341+0000", 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3006e-05 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933350+0000", 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1540000000000008e-06 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933438+0000", 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7661e-05 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933481+0000", 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3089e-05 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.935512+0000", 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020307139999999999 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.935539+0000", 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7078999999999999e-05 2024-09-13T04:58:06.371 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:25 3.8 e27/24 3:173a9b69:::rbd_header.112a8b98d47:head v 27'2, mlcod=0'0)", 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:05.946886+0000", 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.41620731900000002, 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087525000000000005, 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946886+0000", 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946886+0000", 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946885+0000", 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:06.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946915+0000", 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0630999999999999e-05 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946938+0000", 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3244999999999999e-05 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946950+0000", 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2004000000000001e-05 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.947034+0000", 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3620000000000002e-05 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.947070+0000", 2024-09-13T04:58:06.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5666e-05 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.947728+0000", 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065834400000000005 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.947762+0000", 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3507000000000003e-05 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:06.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:06.377 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:06.377 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:06.378 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:06.425 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:06.424+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:06.503 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:06.500+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:06.509 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:06.510 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:06.510 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:06.510 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:06.523 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:06.609 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:06.608+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:06.686 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:06.687 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:06.699 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:06.741 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:06.740+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:06.805 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:06.804+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:06.859 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:06.859 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:06.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:06.859 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:06.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.859 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:10 3.1c e27/24 3:3831950c:::rbd_directory:head v 27'1, mlcod=0'0)", 2024-09-13T04:58:06.859 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:05.925632+0000", 2024-09-13T04:58:06.859 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.93424560599999995, 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012146419999999999, 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925632+0000", 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925632+0000", 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:06.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925632+0000", 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925640+0000", 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1650000000000006e-06 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925659+0000", 2024-09-13T04:58:06.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8906000000000002e-05 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925672+0000", 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3349999999999999e-05 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925762+0000", 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9858999999999995e-05 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:06.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925792+0000", 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9479e-05 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.926797+0000", 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010049220000000001 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.926847+0000", 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0419e-05 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:06.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:13 3.c 3:32c90b40:::rbd_object_map.112a8b98d47:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:05.932786+0000", 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92709190900000005, 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032552359999999999, 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932786+0000", 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932786+0000", 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932784+0000", 2024-09-13T04:58:06.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932795+0000", 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1168e-05 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932814+0000", 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8507000000000001e-05 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:06.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932824+0000", 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5340000000000005e-06 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932881+0000", 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7504999999999999e-05 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.933062+0000", 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018046699999999999 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.933262+0000", 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000200893 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.934204+0000", 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000941606 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935919+0000", 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017147569999999999 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935990+0000", 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0925000000000003e-05 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.936041+0000", 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1548999999999997e-05 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop_reply(client.4394.0:13 3.c e27/24 ondisk, result = 0)", 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:05.935811+0000", 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92406738099999997, 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024327800000000001, 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935811+0000", 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935811+0000", 2024-09-13T04:58:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935809+0000", 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935820+0000", 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0507e-05 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935838+0000", 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8046999999999998e-05 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935846+0000", 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1969999999999996e-06 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935898+0000", 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1576999999999998e-05 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935910+0000", 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2257e-05 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.936054+0000", 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014377300000000001 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:06.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:06.905 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:06.904+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:06.968 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:06.989 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:06.989 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:07.002 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:07.006 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:07.004+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:07.107 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:07.104+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:07.111 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:07.111 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:07.127 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:07.144 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:07.144 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:07.156 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:07.156 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:07.160 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:07.160 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:07.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:07.180 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:07.181 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:07.201 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:07.201 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:07.208 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:07.208+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:07.221 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:07.221 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:07.241 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:07.283 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:07.283 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:07.296 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:07.341 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:07.340+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:07.410 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:07.408+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:07.453 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:07.454 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:07.466 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:07.511 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:07.508+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:07.531 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:58:07.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T04:58:07.586 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:07.586 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:07.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:07.645 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:07.644+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:07.713 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:07.712+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:07.716 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:07.716 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:07.731 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:07.817 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:07.816+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:07.862 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:07.862 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:07.884 INFO:teuthology.orchestra.run.smithi001.stdout:5 write 0x8eeea53 thru 0x8eef703 (0xcb1 bytes) 2024-09-13T04:58:07.889 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:07.889 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:07.891 INFO:teuthology.orchestra.run.smithi001.stdout:6 write 0x37de26f thru 0x37ebc32 (0xd9c4 bytes) 2024-09-13T04:58:07.904 INFO:teuthology.orchestra.run.smithi001.stdout:7 read 0x6ab1277 thru 0x6ab6684 (0x540e bytes) 2024-09-13T04:58:07.905 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:07.905 INFO:teuthology.orchestra.run.smithi001.stdout:8 punch from 0x20c6375 to 0x20c752e, (0x11b9 bytes) 2024-09-13T04:58:07.905 INFO:teuthology.orchestra.run.smithi001.stdout:9 punch from 0x2ab8f6e to 0x2ac0863, (0x78f5 bytes) 2024-09-13T04:58:07.905 INFO:teuthology.orchestra.run.smithi001.stdout:10 write 0x1d395a6 thru 0x1d48430 (0xee8b bytes) 2024-09-13T04:58:07.915 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:07.912+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:07.918 INFO:teuthology.orchestra.run.smithi001.stdout:11 read 0x67eb1da thru 0x67f120f (0x6036 bytes) 2024-09-13T04:58:07.918 INFO:teuthology.orchestra.run.smithi001.stdout:12 write 0x3c42f thru 0x401c2 (0x3d94 bytes) 2024-09-13T04:58:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:07.926 INFO:teuthology.orchestra.run.smithi001.stdout:14 punch from 0x3856286 to 0x38642c1, (0xe03b bytes) 2024-09-13T04:58:07.926 INFO:teuthology.orchestra.run.smithi001.stdout:15 read 0x57a6ca7 thru 0x57b0d9b (0xa0f5 bytes) 2024-09-13T04:58:07.926 INFO:teuthology.orchestra.run.smithi001.stdout:16 clone 1 order 23 su 524288 sc 7 2024-09-13T04:58:07.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:58:08.015 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:08.016+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:08.034 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:08.034 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:08.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:08.117 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:08.116+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:08.182 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:08.182 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:08.195 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:08.217 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T04:58:08.220 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:08.216+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:08.237 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T04:58:08.318 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:08.316+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:08.331 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:08.331 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:08.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:08.419 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:08.416+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:08.426 INFO:teuthology.orchestra.run.smithi001.stdout:17 writesame 0xd2e0ba4 thru 0xd2ef92d data_size 0xf(0xed8a bytes) 2024-09-13T04:58:08.427 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.427 INFO:teuthology.orchestra.run.smithi001.stdout:19 read 0x6054b1 thru 0x60e402 (0x8f52 bytes) 2024-09-13T04:58:08.427 INFO:teuthology.orchestra.run.smithi001.stdout:20 read 0x8fa1bee thru 0x8fa378a (0x1b9d bytes) 2024-09-13T04:58:08.427 INFO:teuthology.orchestra.run.smithi001.stdout:22 read 0x445611d thru 0x4463ec9 (0xddad bytes) 2024-09-13T04:58:08.427 INFO:teuthology.orchestra.run.smithi001.stdout:23 read 0xcec2ad1 thru 0xcec70b7 (0x45e7 bytes) 2024-09-13T04:58:08.427 INFO:teuthology.orchestra.run.smithi001.stdout:24 writesame 0x6316331 thru 0x632391b data_size 0x1(0xd5eb bytes) 2024-09-13T04:58:08.461 INFO:teuthology.orchestra.run.smithi001.stdout:25 read 0x906a092 thru 0x9079fdf (0xff4e bytes) 2024-09-13T04:58:08.462 INFO:teuthology.orchestra.run.smithi001.stdout:28 write 0xc3bc68b thru 0xc3ca270 (0xdbe6 bytes) 2024-09-13T04:58:08.468 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.468 INFO:teuthology.orchestra.run.smithi001.stdout:30 writesame 0x44ce939 thru 0x44dd465 data_size 0x1(0xeb2d bytes) 2024-09-13T04:58:08.511 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:08.511 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:08.517 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.517 INFO:teuthology.orchestra.run.smithi001.stdout:35 punch from 0x25224ca to 0x2527b73, (0x56a9 bytes) 2024-09-13T04:58:08.517 INFO:teuthology.orchestra.run.smithi001.stdout:39 write 0x14f6484 thru 0x14fbaf7 (0x5674 bytes) 2024-09-13T04:58:08.520 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:08.516+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:08.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:08.536 INFO:teuthology.orchestra.run.smithi001.stdout:40 punch from 0xafce7f3 to 0xafd7d21, (0x952e bytes) 2024-09-13T04:58:08.537 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.537 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.540 INFO:teuthology.orchestra.run.smithi001.stdout:45 write 0x48040e thru 0x48cbb9 (0xc7ac bytes) 2024-09-13T04:58:08.557 INFO:teuthology.orchestra.run.smithi001.stdout:46 write 0x32d11 thru 0x3cb27 (0x9e17 bytes) 2024-09-13T04:58:08.570 INFO:teuthology.orchestra.run.smithi001.stdout:47 punch from 0x16e395c to 0x16f2009, (0xe6ad bytes) 2024-09-13T04:58:08.582 INFO:teuthology.orchestra.run.smithi001.stdout:48 write 0xdc05d8c thru 0xdc10740 (0xa9b5 bytes) 2024-09-13T04:58:08.585 INFO:teuthology.orchestra.run.smithi001.stdout:50 read 0x3174afb thru 0x317d5af (0x8ab5 bytes) 2024-09-13T04:58:08.587 INFO:teuthology.orchestra.run.smithi001.stdout:51 read 0xb6f9abe thru 0xb6fe063 (0x45a6 bytes) 2024-09-13T04:58:08.587 INFO:teuthology.orchestra.run.smithi001.stdout:52 write 0x668affc thru 0x6691f3e (0x6f43 bytes) 2024-09-13T04:58:08.592 INFO:teuthology.orchestra.run.smithi001.stdout:53 read 0x2b40266 thru 0x2b45abf (0x585a bytes) 2024-09-13T04:58:08.593 INFO:teuthology.orchestra.run.smithi001.stdout:54 read 0x9e935c6 thru 0x9e9c4aa (0x8ee5 bytes) 2024-09-13T04:58:08.593 INFO:teuthology.orchestra.run.smithi001.stdout:55 read 0x7b0b8ec thru 0x7b19093 (0xd7a8 bytes) 2024-09-13T04:58:08.593 INFO:teuthology.orchestra.run.smithi001.stdout:57 read 0x95f9854 thru 0x9607ce0 (0xe48d bytes) 2024-09-13T04:58:08.594 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.594 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0x7544a62 2024-09-13T04:58:08.607 INFO:teuthology.orchestra.run.smithi001.stdout:59 trunc from 0xdc10741 to 0x7544a62 2024-09-13T04:58:08.620 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:08.616+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:08.639 INFO:teuthology.orchestra.run.smithi001.stdout:60 write 0x1dc1ec1 thru 0x1dc2f1e (0x105e bytes) 2024-09-13T04:58:08.644 INFO:teuthology.orchestra.run.smithi001.stdout:61 punch from 0x1c24ff5 to 0x1c2e0c7, (0x90d2 bytes) 2024-09-13T04:58:08.644 INFO:teuthology.orchestra.run.smithi001.stdout:62 punch from 0x24460c0 to 0x245356b, (0xd4ab bytes) 2024-09-13T04:58:08.644 INFO:teuthology.orchestra.run.smithi001.stdout:63 write 0x3a4a189 thru 0x3a50e62 (0x6cda bytes) 2024-09-13T04:58:08.647 INFO:teuthology.orchestra.run.smithi001.stdout:65 write 0x66065b6 thru 0x660c168 (0x5bb3 bytes) 2024-09-13T04:58:08.649 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:08.649 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:08.653 INFO:teuthology.orchestra.run.smithi001.stdout:66 read 0x32e03b0 thru 0x32ef8a1 (0xf4f2 bytes) 2024-09-13T04:58:08.653 INFO:teuthology.orchestra.run.smithi001.stdout:67 punch from 0x4d9acb0 to 0x4da0ba0, (0x5ef0 bytes) 2024-09-13T04:58:08.654 INFO:teuthology.orchestra.run.smithi001.stdout:69 write 0x1d068c0 thru 0x1d117a1 (0xaee2 bytes) 2024-09-13T04:58:08.663 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:08.673 INFO:teuthology.orchestra.run.smithi001.stdout:71 write 0x8fba70b thru 0x8fc09a4 (0x629a bytes) 2024-09-13T04:58:08.693 INFO:teuthology.orchestra.run.smithi001.stdout:72 write 0xc34acd8 thru 0xc350805 (0x5b2e bytes) 2024-09-13T04:58:08.700 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.700 INFO:teuthology.orchestra.run.smithi001.stdout:74 writesame 0x7c774a5 thru 0x7c78370 data_size 0x2(0xecc bytes) 2024-09-13T04:58:08.704 INFO:teuthology.orchestra.run.smithi001.stdout:75 write 0x318f1d4 thru 0x3196e7c (0x7ca9 bytes) 2024-09-13T04:58:08.712 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.712 INFO:teuthology.orchestra.run.smithi001.stdout:79 writesame 0xbbe5d36 thru 0xbbe9171 data_size 0x2(0x343c bytes) 2024-09-13T04:58:08.721 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:08.720+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:08.726 INFO:teuthology.orchestra.run.smithi001.stdout:80 read 0x2873c9 thru 0x2930d5 (0xbd0d bytes) 2024-09-13T04:58:08.727 INFO:teuthology.orchestra.run.smithi001.stdout:82 read 0x76dda02 thru 0x76ea101 (0xc700 bytes) 2024-09-13T04:58:08.739 INFO:teuthology.orchestra.run.smithi001.stdout:83 trunc from 0xc350806 to 0x74ab8bd 2024-09-13T04:58:08.766 INFO:teuthology.orchestra.run.smithi001.stdout:84 punch from 0x5b1d8f4 to 0x5b24cd3, (0x73df bytes) 2024-09-13T04:58:08.766 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.766 INFO:teuthology.orchestra.run.smithi001.stdout:87 writesame 0xc2bbc7 thru 0xc31c8d data_size 0x5(0x60c7 bytes) 2024-09-13T04:58:08.777 INFO:teuthology.orchestra.run.smithi001.stdout:88 writesame 0x28125fe thru 0x281dc83 data_size 0x3d(0xb686 bytes) 2024-09-13T04:58:08.782 INFO:teuthology.orchestra.run.smithi001.stdout:89 punch from 0x5453ccd to 0x5453dd3, (0x106 bytes) 2024-09-13T04:58:08.782 INFO:teuthology.orchestra.run.smithi001.stdout:90 read 0x567470 thru 0x568de4 (0x1975 bytes) 2024-09-13T04:58:08.782 INFO:teuthology.orchestra.run.smithi001.stdout:91 read 0xc07cf0 thru 0xc119e6 (0x9cf7 bytes) 2024-09-13T04:58:08.788 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:08.788 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:08.802 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:08.811 INFO:teuthology.orchestra.run.smithi001.stdout:92 write 0xd882d1b thru 0xd88358f (0x875 bytes) 2024-09-13T04:58:08.817 INFO:teuthology.orchestra.run.smithi001.stdout:94 punch from 0x1973fdd to 0x19793cd, (0x53f0 bytes) 2024-09-13T04:58:08.817 INFO:teuthology.orchestra.run.smithi001.stdout:96 read 0x3c82321 thru 0x3c8f249 (0xcf29 bytes) 2024-09-13T04:58:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:97 read 0x362f25f thru 0x3636bf5 (0x7997 bytes) 2024-09-13T04:58:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:101 writesame 0x9a866a1 thru 0x9a9600a data_size 0x19(0xf96a bytes) 2024-09-13T04:58:08.822 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:08.820+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:08.826 INFO:teuthology.orchestra.run.smithi001.stdout:102 punch from 0x931a4b3 to 0x931c57a, (0x20c7 bytes) 2024-09-13T04:58:08.826 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:08.827 INFO:teuthology.orchestra.run.smithi001.stdout:105 writesame 0x792ce94 thru 0x793b958 data_size 0x1(0xeac5 bytes) 2024-09-13T04:58:08.921 INFO:teuthology.orchestra.run.smithi001.stdout:106 punch from 0x9e0f64a to 0x9e1307b, (0x3a31 bytes) 2024-09-13T04:58:08.923 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:08.920+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:08.937 INFO:teuthology.orchestra.run.smithi001.stdout:108 write 0xec2a615 thru 0xec37530 (0xcf1c bytes) 2024-09-13T04:58:08.962 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:08.963 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:08.966 INFO:teuthology.orchestra.run.smithi001.stdout:109 trunc from 0xec37531 to 0x640508e 2024-09-13T04:58:08.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:09.023 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:09.020+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:09.046 INFO:teuthology.orchestra.run.smithi001.stdout:111 writesame 0xd406887 thru 0xd408a0d data_size 0x1(0x2187 bytes) 2024-09-13T04:58:09.047 INFO:teuthology.orchestra.run.smithi001.stdout:112 writesame 0xa91ff62 thru 0xa92c01f data_size 0x1(0xc0be bytes) 2024-09-13T04:58:09.096 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:09.096 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:09.112 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:09.126 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xd974b03 2024-09-13T04:58:09.127 INFO:teuthology.orchestra.run.smithi001.stdout:113 trunc from 0xd408a0e to 0xd974b03 2024-09-13T04:58:09.145 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.157 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:09.156+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:09.177 INFO:teuthology.orchestra.run.smithi001.stdout:115 trunc from 0xd974b03 to 0x1a8c1e7 2024-09-13T04:58:09.220 INFO:teuthology.orchestra.run.smithi001.stdout:116 punch from 0x118588a to 0x118cdec, (0x7562 bytes) 2024-09-13T04:58:09.220 INFO:teuthology.orchestra.run.smithi001.stdout:119 read 0x1154105 thru 0x115e967 (0xa863 bytes) 2024-09-13T04:58:09.221 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.225 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:09.224+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:09.238 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:58:09.239 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T04:58:09.247 INFO:teuthology.orchestra.run.smithi001.stdout:121 write 0x551a01e thru 0x5525c06 (0xbbe9 bytes) 2024-09-13T04:58:09.258 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:09.269 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T04:58:09.269 INFO:teuthology.orchestra.run.smithi001.stdout:122 writesame 0xc2aaa42 thru 0xc2b516d data_size 0xd(0xa72c bytes) 2024-09-13T04:58:09.269 INFO:teuthology.orchestra.run.smithi001.stdout:123 read 0x9761071 thru 0x97684c1 (0x7451 bytes) 2024-09-13T04:58:09.270 INFO:teuthology.orchestra.run.smithi001.stdout:124 trunc from 0xc2b516e to 0xbfc60cc 2024-09-13T04:58:09.275 INFO:teuthology.orchestra.run.smithi001.stdout:125 punch from 0x7c8546c to 0x7c917da, (0xc36e bytes) 2024-09-13T04:58:09.275 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:09.290 INFO:teuthology.orchestra.run.smithi001.stdout:127 trunc from 0xbfc60cc to 0x1ef96bb 2024-09-13T04:58:09.292 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:09.312 INFO:teuthology.orchestra.run.smithi001.stdout:128 write 0x1685d4e thru 0x16897e2 (0x3a95 bytes) 2024-09-13T04:58:09.326 INFO:teuthology.orchestra.run.smithi001.stdout:130 writesame 0x6481e91 thru 0x648b053 data_size 0x11(0x91c3 bytes) 2024-09-13T04:58:09.326 INFO:teuthology.orchestra.run.smithi001.stdout:131 write 0x1778f3e thru 0x177fff5 (0x70b8 bytes) 2024-09-13T04:58:09.331 INFO:teuthology.orchestra.run.smithi001.stdout:133 punch from 0x3ac66d7 to 0x3ad60d4, (0xf9fd bytes) 2024-09-13T04:58:09.331 INFO:teuthology.orchestra.run.smithi001.stdout:134 write 0x8dd0aa thru 0x8eadbf (0xdd16 bytes) 2024-09-13T04:58:09.336 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:09.336+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:09.343 INFO:teuthology.orchestra.run.smithi001.stdout:135 writesame 0x6d09bb7 thru 0x6d0dfb1 data_size 0x1(0x43fb bytes) 2024-09-13T04:58:09.385 INFO:teuthology.orchestra.run.smithi001.stdout:136 write 0xa168fb6 thru 0xa17717c (0xe1c7 bytes) 2024-09-13T04:58:09.385 INFO:teuthology.orchestra.run.smithi001.stdout:137 read 0x1fbf669 thru 0x1fc9241 (0x9bd9 bytes) 2024-09-13T04:58:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:138 write 0x5c0e482 thru 0x5c11346 (0x2ec5 bytes) 2024-09-13T04:58:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:140 write 0xd2a0e02 thru 0xd2b011b (0xf31a bytes) 2024-09-13T04:58:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:141 read 0x6d149f9 thru 0x6d19da7 (0x53af bytes) 2024-09-13T04:58:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:142 read 0x97167a6 thru 0x9717886 (0x10e1 bytes) 2024-09-13T04:58:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:143 write 0x313f6d6 thru 0x3146b57 (0x7482 bytes) 2024-09-13T04:58:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.386 INFO:teuthology.orchestra.run.smithi001.stdout:145 trunc from 0xd2b011c to 0xa099195 2024-09-13T04:58:09.399 INFO:teuthology.orchestra.run.smithi001.stdout:146 punch from 0x9aa697b to 0x9ab3836, (0xcebb bytes) 2024-09-13T04:58:09.399 INFO:teuthology.orchestra.run.smithi001.stdout:147 punch from 0x65848dc to 0x6587e95, (0x35b9 bytes) 2024-09-13T04:58:09.399 INFO:teuthology.orchestra.run.smithi001.stdout:148 write 0x3862346 thru 0x3864c6f (0x292a bytes) 2024-09-13T04:58:09.413 INFO:teuthology.orchestra.run.smithi001.stdout:150 trunc from 0xa099195 to 0x2af8307 2024-09-13T04:58:09.424 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:09.425 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:09.427 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:09.424+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:09.429 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.429 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.430 INFO:teuthology.orchestra.run.smithi001.stdout:155 read 0x575cad thru 0x5813b7 (0xb70b bytes) 2024-09-13T04:58:09.430 INFO:teuthology.orchestra.run.smithi001.stdout:156 punch from 0x19c7584 to 0x19d0e13, (0x988f bytes) 2024-09-13T04:58:09.431 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.445 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:09.445 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:09.454 INFO:teuthology.orchestra.run.smithi001.stdout:161 write 0xe3d974f thru 0xe3df27e (0x5b30 bytes) 2024-09-13T04:58:09.458 INFO:teuthology.orchestra.run.smithi001.stdout:162 writesame 0x678d690 thru 0x67923be data_size 0x1(0x4d2f bytes) 2024-09-13T04:58:09.460 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:09.460 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:09.468 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:09.468 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:09.472 INFO:teuthology.orchestra.run.smithi001.stdout:163 writesame 0x9001494 thru 0x900e8af data_size 0x64(0xd41c bytes) 2024-09-13T04:58:09.474 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:09.477 INFO:teuthology.orchestra.run.smithi001.stdout:167 read 0xde3573c thru 0xde40985 (0xb24a bytes) 2024-09-13T04:58:09.478 INFO:teuthology.orchestra.run.smithi001.stdout:168 write 0x97a6eb5 thru 0x97ab900 (0x4a4c bytes) 2024-09-13T04:58:09.488 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:09.488 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:09.499 INFO:teuthology.orchestra.run.smithi001.stdout:169 trunc from 0xe3df27f to 0x24e9093 2024-09-13T04:58:09.509 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:09.528 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:09.528 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:09.528 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:09.528+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:09.532 INFO:teuthology.orchestra.run.smithi001.stdout:170 write 0xa784ffb thru 0xa78cbd4 (0x7bda bytes) 2024-09-13T04:58:09.536 INFO:teuthology.orchestra.run.smithi001.stdout:172 write 0x51e55e4 thru 0x51f2f27 (0xd944 bytes) 2024-09-13T04:58:09.542 INFO:teuthology.orchestra.run.smithi001.stdout:173 read 0x4b0f06e thru 0x4b15f5b (0x6eee bytes) 2024-09-13T04:58:09.547 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:09.548 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:09.556 INFO:teuthology.orchestra.run.smithi001.stdout:176 trunc from 0xa78cbd5 to 0xf955e4 2024-09-13T04:58:09.570 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:09.574 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.574 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.592 INFO:teuthology.orchestra.run.smithi001.stdout:180 write 0x8ceb3b9 thru 0x8cfb294 (0xfedc bytes) 2024-09-13T04:58:09.597 INFO:teuthology.orchestra.run.smithi001.stdout:181 read 0x48cac92 thru 0x48d481e (0x9b8d bytes) 2024-09-13T04:58:09.600 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4245.0:5 2.2 e22/20 2:5cea7035:::rbd_info:head v 22'1, mlcod=0'0)", 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:55.159081+0000", 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.442556376000001, 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002107023, 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:09.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159081+0000", 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159081+0000", 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159081+0000", 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:09.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159089+0000", 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9099999999999994e-06 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159117+0000", 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7653000000000001e-05 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:09.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159142+0000", 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4559999999999999e-05 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159267+0000", 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000125143 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.159349+0000", 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1961999999999999e-05 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:09.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.161124+0000", 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001774905 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:55.161188+0000", 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4423000000000003e-05 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4245.0:7 2.2 e23/20 2:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-13T04:58:09.605 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:57:56.158816+0000", 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.442821851, 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020319090000000001, 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158816+0000", 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158816+0000", 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:09.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158815+0000", 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158824+0000", 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9670000000000007e-06 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:09.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158852+0000", 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8183000000000001e-05 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158867+0000", 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4316e-05 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.158984+0000", 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000116989 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.159056+0000", 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2862999999999997e-05 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.160779+0000", 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001722105 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:57:56.160848+0000", 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8992999999999995e-05 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:09.609 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:09.610 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:09.610 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:09.610 INFO:teuthology.orchestra.run.smithi001.stdout:184 write 0x9587720 thru 0x95940b5 (0xc996 bytes) 2024-09-13T04:58:09.615 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:09.620 INFO:teuthology.orchestra.run.smithi001.stdout:186 write 0xdd86fa7 thru 0xdd90be3 (0x9c3d bytes) 2024-09-13T04:58:09.624 INFO:teuthology.orchestra.run.smithi001.stdout:187 write 0xd6e933c thru 0xd6f2b93 (0x9858 bytes) 2024-09-13T04:58:09.626 INFO:teuthology.orchestra.run.smithi001.stdout:188 writesame 0xcd0debf thru 0xcd17a83 data_size 0x1(0x9bc5 bytes) 2024-09-13T04:58:09.628 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:09.628+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:09.650 INFO:teuthology.orchestra.run.smithi001.stdout:192 read 0x3c4d2b4 thru 0x3c4d670 (0x3bd bytes) 2024-09-13T04:58:09.650 INFO:teuthology.orchestra.run.smithi001.stdout:193 write 0x4ae077a thru 0x4aec55e (0xbde5 bytes) 2024-09-13T04:58:09.656 INFO:teuthology.orchestra.run.smithi001.stdout:196 write 0x62e74d6 thru 0x62eff5e (0x8a89 bytes) 2024-09-13T04:58:09.662 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.662 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.662 INFO:teuthology.orchestra.run.smithi001.stdout:199 read 0xc1289ac thru 0xc12c427 (0x3a7c bytes) 2024-09-13T04:58:09.662 INFO:teuthology.orchestra.run.smithi001.stdout:200 read 0x8ac52bc thru 0x8acae50 (0x5b95 bytes) 2024-09-13T04:58:09.678 INFO:teuthology.orchestra.run.smithi001.stdout:202 trunc from 0xdd90be4 to 0x43afec0 2024-09-13T04:58:09.721 INFO:teuthology.orchestra.run.smithi001.stdout:204 write 0xd2ec40a thru 0xd2f99b2 (0xd5a9 bytes) 2024-09-13T04:58:09.726 INFO:teuthology.orchestra.run.smithi001.stdout:205 read 0xa0b0312 thru 0xa0b8f1c (0x8c0b bytes) 2024-09-13T04:58:09.726 INFO:teuthology.orchestra.run.smithi001.stdout:207 read 0x90f5392 thru 0x910291e (0xd58d bytes) 2024-09-13T04:58:09.726 INFO:teuthology.orchestra.run.smithi001.stdout:208 write 0x5c3e89c thru 0x5c44cf7 (0x645c bytes) 2024-09-13T04:58:09.729 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:09.728+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:09.731 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.731 INFO:teuthology.orchestra.run.smithi001.stdout:210 write 0xb2c0b01 thru 0xb2cefa4 (0xe4a4 bytes) 2024-09-13T04:58:09.760 INFO:teuthology.orchestra.run.smithi001.stdout:214 trunc from 0xd2f99b3 to 0x215df6a 2024-09-13T04:58:09.783 INFO:teuthology.orchestra.run.smithi001.stdout:215 write 0x1ffc87e thru 0x20007ba (0x3f3d bytes) 2024-09-13T04:58:09.785 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:09.785 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:09.785 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:09.785 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:09.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:09.805 INFO:teuthology.orchestra.run.smithi001.stdout:216 write 0x563fa27 thru 0x564eb2c (0xf106 bytes) 2024-09-13T04:58:09.823 INFO:teuthology.orchestra.run.smithi001.stdout:217 write 0x6372d22 thru 0x6382947 (0xfc26 bytes) 2024-09-13T04:58:09.831 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:09.828+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:09.831 INFO:teuthology.orchestra.run.smithi001.stdout:218 read 0xe16a0d thru 0xe23ee5 (0xd4d9 bytes) 2024-09-13T04:58:09.832 INFO:teuthology.orchestra.run.smithi001.stdout:220 writesame 0x2dc369e thru 0x2dca203 data_size 0x3b(0x6b66 bytes) 2024-09-13T04:58:09.836 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:09.836 INFO:teuthology.orchestra.run.smithi001.stdout:224 punch from 0xf8f47a to 0xf9dfc0, (0xeb46 bytes) 2024-09-13T04:58:09.836 INFO:teuthology.orchestra.run.smithi001.stdout:228 read 0x53ca56c thru 0x53d4fb8 (0xaa4d bytes) 2024-09-13T04:58:09.836 INFO:teuthology.orchestra.run.smithi001.stdout:229 read 0x4d2ad5c thru 0x4d30e8e (0x6133 bytes) 2024-09-13T04:58:09.836 INFO:teuthology.orchestra.run.smithi001.stdout:230 writesame 0x3a2e527 thru 0x3a387f3 data_size 0x1(0xa2cd bytes) 2024-09-13T04:58:09.917 INFO:teuthology.orchestra.run.smithi001.stdout:231 write 0xd731574 thru 0xd73f446 (0xded3 bytes) 2024-09-13T04:58:09.922 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:09.922 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:09.922 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:09.922 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:09.922 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:09.925 INFO:teuthology.orchestra.run.smithi001.stdout:232 write 0x507e51a thru 0x50855b8 (0x709f bytes) 2024-09-13T04:58:09.931 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:09.932+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:09.936 INFO:teuthology.orchestra.run.smithi001.stdout:234 punch from 0x2629ffe to 0x2638bc3, (0xebc5 bytes) 2024-09-13T04:58:09.936 INFO:teuthology.orchestra.run.smithi001.stdout:235 write 0x82c6709 thru 0x82c75a4 (0xe9c bytes) 2024-09-13T04:58:09.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:09.960 INFO:teuthology.orchestra.run.smithi001.stdout:236 trunc from 0xd73f447 to 0x71914b7 2024-09-13T04:58:10.003 INFO:teuthology.orchestra.run.smithi001.stdout:238 writesame 0x76ed204 thru 0x76fa09b data_size 0x16(0xce98 bytes) 2024-09-13T04:58:10.022 INFO:teuthology.orchestra.run.smithi001.stdout:240 trunc from 0x76fa09c to 0x9927b2 2024-09-13T04:58:10.033 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:10.032+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:10.054 INFO:teuthology.orchestra.run.smithi001.stdout:241 trunc from 0x9927b2 to 0x20abede 2024-09-13T04:58:10.063 INFO:teuthology.orchestra.run.smithi001.stdout:242 read 0x15360b4 thru 0x153e898 (0x87e5 bytes) 2024-09-13T04:58:10.078 INFO:teuthology.orchestra.run.smithi001.stdout:243 trunc from 0x20abede to 0x725dae1 2024-09-13T04:58:10.090 INFO:teuthology.orchestra.run.smithi001.stdout:244 writesame 0x339b089 thru 0x33a417d data_size 0x1(0x90f5 bytes) 2024-09-13T04:58:10.108 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:10.108 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:10.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:10.108 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:10.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.108 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:12 3.8 e27/24 3:173a9b69:::rbd_header.112a8b98d47:head v 27'1, mlcod=0'0)", 2024-09-13T04:58:10.108 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:05.931015+0000", 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.1773386730000004, 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001284758, 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931015+0000", 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931015+0000", 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931015+0000", 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931019+0000", 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2239999999999997e-06 2024-09-13T04:58:10.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931028+0000", 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1579999999999994e-06 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931037+0000", 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0000000000000002e-06 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:10.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931100+0000", 2024-09-13T04:58:10.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2998999999999998e-05 2024-09-13T04:58:10.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:10.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.931133+0000", 2024-09-13T04:58:10.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3203e-05 2024-09-13T04:58:10.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.932263+0000", 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001129583 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.932300+0000", 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6912999999999997e-05 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.113 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:13 3.c e27/24 3:32c90b40:::rbd_object_map.112a8b98d47:head v 27'1, mlcod=0'0)", 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:05.933323+0000", 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.1750305670000003, 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0022159739999999999, 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933323+0000", 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933323+0000", 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933322+0000", 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933328+0000", 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8649999999999998e-06 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933341+0000", 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3006e-05 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933350+0000", 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1540000000000008e-06 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933438+0000", 2024-09-13T04:58:10.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7661e-05 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.933481+0000", 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3089e-05 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.935512+0000", 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0020307139999999999 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.935539+0000", 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7078999999999999e-05 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:25 3.8 e27/24 3:173a9b69:::rbd_header.112a8b98d47:head v 27'2, mlcod=0'0)", 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:05.946886+0000", 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.1614672590000001, 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00087525000000000005, 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:10.118 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946886+0000", 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946886+0000", 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946885+0000", 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946915+0000", 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0630999999999999e-05 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946938+0000", 2024-09-13T04:58:10.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3244999999999999e-05 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.946950+0000", 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2004000000000001e-05 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.947034+0000", 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3620000000000002e-05 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:10.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.947070+0000", 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5666e-05 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.947728+0000", 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00065834400000000005 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:05.947762+0000", 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3507000000000003e-05 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.122 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:656 3.0 e29/24 3:0370c3a4:::rbd_data.112ac9124348.000000000000000f:head v 29'7, mlcod=29'5)", 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:09.596277+0000", 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.51207617299999997, 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024158529999999999, 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:10.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.596277+0000", 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.596277+0000", 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.596275+0000", 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:10.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.596380+0000", 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000104979 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.596410+0000", 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9784000000000001e-05 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.596430+0000", 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0602999999999999e-05 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.596514+0000", 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4052000000000001e-05 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.596554+0000", 2024-09-13T04:58:10.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9342999999999999e-05 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.598617+0000", 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002063283 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.598693+0000", 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6196e-05 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:687 3.0 e29/24 3:0370c3a4:::rbd_data.112ac9124348.000000000000000f:head v 29'8, mlcod=29'6)", 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:09.684362+0000", 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.42399133100000003, 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000780476, 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.684362+0000", 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.684362+0000", 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.684361+0000", 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.684373+0000", 2024-09-13T04:58:10.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1874e-05 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.684394+0000", 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1766000000000001e-05 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.684403+0000", 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5909999999999996e-06 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.684475+0000", 2024-09-13T04:58:10.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2071999999999997e-05 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.684498+0000", 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3025999999999999e-05 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.685078+0000", 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058028099999999998 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:10.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:09.685143+0000", 2024-09-13T04:58:10.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4508e-05 2024-09-13T04:58:10.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.132 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:10.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:10.132 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:10.132 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:10.133 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:10.134 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:10.132+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:10.152 INFO:teuthology.orchestra.run.smithi001.stdout:245 read 0x7053918 thru 0x705cfb2 (0x969b bytes) 2024-09-13T04:58:10.166 INFO:teuthology.orchestra.run.smithi001.stdout:246 writesame 0x90def62 thru 0x90e0b78 data_size 0x2f(0x1c17 bytes) 2024-09-13T04:58:10.167 INFO:teuthology.orchestra.run.smithi001.stdout:247 writesame 0x54cabaf thru 0x54d9966 data_size 0x4(0xedb8 bytes) 2024-09-13T04:58:10.186 INFO:teuthology.orchestra.run.smithi001.stdout:248 trunc from 0x90e0b79 to 0x5fc5292 2024-09-13T04:58:10.199 INFO:teuthology.orchestra.run.smithi001.stdout:249 read 0x33e7783 thru 0x33f0f25 (0x97a3 bytes) 2024-09-13T04:58:10.200 INFO:teuthology.orchestra.run.smithi001.stdout:250 read 0x48bcc1c thru 0x48c75c4 (0xa9a9 bytes) 2024-09-13T04:58:10.200 INFO:teuthology.orchestra.run.smithi001.stdout:251 write 0x97a6df thru 0x982b24 (0x8446 bytes) 2024-09-13T04:58:10.208 INFO:teuthology.orchestra.run.smithi001.stdout:255 trunc from 0x5fc5292 to 0x768ba26 2024-09-13T04:58:10.215 INFO:teuthology.orchestra.run.smithi001.stdout:256 writesame 0x56f11cf thru 0x56f754d data_size 0x1(0x637f bytes) 2024-09-13T04:58:10.234 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:10.232+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T04:58:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.253 INFO:teuthology.orchestra.run.smithi001.stdout:261 trunc from 0x768ba26 to 0x3c5cff6 2024-09-13T04:58:10.270 INFO:teuthology.orchestra.run.smithi001.stdout:262 trunc from 0x3c5cff6 to 0x12c4c45 2024-09-13T04:58:10.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T04:58:10.299 INFO:teuthology.orchestra.run.smithi001.stdout:263 punch from 0xe13b1b to 0xe21f9c, (0xe481 bytes) 2024-09-13T04:58:10.302 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:10.302 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:10.302 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:10.303 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:10.309 INFO:teuthology.orchestra.run.smithi001.stdout:264 writesame 0x3107332 thru 0x31153bb data_size 0x29(0xe08a bytes) 2024-09-13T04:58:10.311 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.311 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.323 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:10.325 INFO:teuthology.orchestra.run.smithi001.stdout:267 write 0x74447f7 thru 0x744d741 (0x8f4b bytes) 2024-09-13T04:58:10.332 INFO:teuthology.orchestra.run.smithi001.stdout:270 read 0x5bd4ab2 thru 0x5bd6477 (0x19c6 bytes) 2024-09-13T04:58:10.332 INFO:teuthology.orchestra.run.smithi001.stdout:272 read 0x5f8a0ed thru 0x5f949ae (0xa8c2 bytes) 2024-09-13T04:58:10.335 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:10.332+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:10.344 INFO:teuthology.orchestra.run.smithi001.stdout:273 writesame 0xab607b8 thru 0xab65251 data_size 0x9(0x4a9a bytes) 2024-09-13T04:58:10.344 INFO:teuthology.orchestra.run.smithi001.stdout:275 writesame 0x57a56c4 thru 0x57b2d09 data_size 0x1(0xd646 bytes) 2024-09-13T04:58:10.400 INFO:teuthology.orchestra.run.smithi001.stdout:276 trunc from 0xab65252 to 0x38b5410 2024-09-13T04:58:10.420 INFO:teuthology.orchestra.run.smithi001.stdout:277 punch from 0x199c4b9 to 0x19a9738, (0xd27f bytes) 2024-09-13T04:58:10.427 INFO:teuthology.orchestra.run.smithi001.stdout:278 trunc from 0x38b5410 to 0x81a4150 2024-09-13T04:58:10.435 INFO:teuthology.orchestra.run.smithi001.stdout:280 punch from 0x25b903 to 0x266160, (0xa85d bytes) 2024-09-13T04:58:10.435 INFO:teuthology.orchestra.run.smithi001.stdout:281 write 0x62a8e07 thru 0x62b1d52 (0x8f4c bytes) 2024-09-13T04:58:10.435 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:10.436+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:10.440 INFO:teuthology.orchestra.run.smithi001.stdout:282 read 0x6c71b72 thru 0x6c78570 (0x69ff bytes) 2024-09-13T04:58:10.440 INFO:teuthology.orchestra.run.smithi001.stdout:283 read 0x6b5131c thru 0x6b58c51 (0x7936 bytes) 2024-09-13T04:58:10.450 INFO:teuthology.orchestra.run.smithi001.stdout:284 write 0xaf44dea thru 0xaf4ced5 (0x80ec bytes) 2024-09-13T04:58:10.456 INFO:teuthology.orchestra.run.smithi001.stdout:286 writesame 0x94d74e8 thru 0x94e007d data_size 0x11(0x8b96 bytes) 2024-09-13T04:58:10.464 INFO:teuthology.orchestra.run.smithi001.stdout:288 punch from 0x99332ca to 0x99427dc, (0xf512 bytes) 2024-09-13T04:58:10.464 INFO:teuthology.orchestra.run.smithi001.stdout:289 writesame 0x3f52211 thru 0x3f5952c data_size 0x4(0x731c bytes) 2024-09-13T04:58:10.468 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:10.468 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:10.469 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:10.469 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:10.469 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:10.470 INFO:teuthology.orchestra.run.smithi001.stdout:291 read 0x6144e91 thru 0x61532a8 (0xe418 bytes) 2024-09-13T04:58:10.470 INFO:teuthology.orchestra.run.smithi001.stdout:292 writesame 0x6ba6c9 thru 0x6bf016 data_size 0xb(0x494e bytes) 2024-09-13T04:58:10.474 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.474 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.474 INFO:teuthology.orchestra.run.smithi001.stdout:296 writesame 0x4696765 thru 0x46a3641 data_size 0x1(0xcedd bytes) 2024-09-13T04:58:10.485 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:10.513 INFO:teuthology.orchestra.run.smithi001.stdout:299 read 0x2ba4b5f thru 0x2ba7e66 (0x3308 bytes) 2024-09-13T04:58:10.515 INFO:teuthology.orchestra.run.smithi001.stdout:300 writesame 0x6fb49ad thru 0x6fbe2d1 data_size 0x1(0x9925 bytes) 2024-09-13T04:58:10.537 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:10.536+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:10.548 INFO:teuthology.orchestra.run.smithi001.stdout:302 read 0xa4348d9 thru 0xa4420b4 (0xd7dc bytes) 2024-09-13T04:58:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:304 trunc from 0xaf4ced6 to 0xba04e57 2024-09-13T04:58:10.560 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.571 INFO:teuthology.orchestra.run.smithi001.stdout:307 writesame 0xd97a490 thru 0xd98270a data_size 0x1(0x827b bytes) 2024-09-13T04:58:10.577 INFO:teuthology.orchestra.run.smithi001.stdout:308 trunc from 0xd98270b to 0xa9d9fdc 2024-09-13T04:58:10.594 INFO:teuthology.orchestra.run.smithi001.stdout:309 read 0xa7d224b thru 0xa7dee03 (0xcbb9 bytes) 2024-09-13T04:58:10.595 INFO:teuthology.orchestra.run.smithi001.stdout:310 writesame 0x51a5434 thru 0x51b0b53 data_size 0x50(0xb720 bytes) 2024-09-13T04:58:10.599 INFO:teuthology.orchestra.run.smithi001.stdout:312 read 0x52d12c1 thru 0x52d156f (0x2af bytes) 2024-09-13T04:58:10.599 INFO:teuthology.orchestra.run.smithi001.stdout:313 read 0x526b20b thru 0x5275a8a (0xa880 bytes) 2024-09-13T04:58:10.609 INFO:teuthology.orchestra.run.smithi001.stdout:314 trunc from 0xa9d9fdc to 0x542422a 2024-09-13T04:58:10.635 INFO:teuthology.orchestra.run.smithi001.stdout:316 write 0x5f3ebaf thru 0x5f49c4b (0xb09d bytes) 2024-09-13T04:58:10.637 INFO:teuthology.orchestra.run.smithi001.stdout:317 read 0x7262c3 thru 0x72a524 (0x4262 bytes) 2024-09-13T04:58:10.638 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:10.636+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:10.638 INFO:teuthology.orchestra.run.smithi001.stdout:318 read 0xf50e8d thru 0xf5db26 (0xcc9a bytes) 2024-09-13T04:58:10.648 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:10.649 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:10.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:10.649 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:10.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.649 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:10 3.1c e27/24 3:3831950c:::rbd_directory:head v 27'1, mlcod=0'0)", 2024-09-13T04:58:10.649 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:05.925632+0000", 2024-09-13T04:58:10.649 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.7222976469999995, 2024-09-13T04:58:10.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012146419999999999, 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925632+0000", 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925632+0000", 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925632+0000", 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925640+0000", 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1650000000000006e-06 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925659+0000", 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8906000000000002e-05 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925672+0000", 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3349999999999999e-05 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925762+0000", 2024-09-13T04:58:10.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9858999999999995e-05 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.925792+0000", 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9479e-05 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.926797+0000", 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010049220000000001 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.926847+0000", 2024-09-13T04:58:10.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0419e-05 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:13 3.c 3:32c90b40:::rbd_object_map.112a8b98d47:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:05.932786+0000", 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.7151439499999999, 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032552359999999999, 2024-09-13T04:58:10.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932786+0000", 2024-09-13T04:58:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932786+0000", 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932784+0000", 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932795+0000", 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1168e-05 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932814+0000", 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8507000000000001e-05 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932824+0000", 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5340000000000005e-06 2024-09-13T04:58:10.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.932881+0000", 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7504999999999999e-05 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.933062+0000", 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018046699999999999 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.933262+0000", 2024-09-13T04:58:10.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000200893 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.934204+0000", 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000941606 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935919+0000", 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017147569999999999 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.935990+0000", 2024-09-13T04:58:10.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0925000000000003e-05 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:05.936041+0000", 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1548999999999997e-05 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:623 3.e e29/24 3:72ba23be:::rbd_data.112ac9124348.0000000000000014:head v 29'9, mlcod=28'7)", 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.536299+0000", 2024-09-13T04:58:10.660 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.1116309209999999, 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001092165, 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.536299+0000", 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.536299+0000", 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.536299+0000", 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.536326+0000", 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6917e-05 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.536338+0000", 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1773000000000001e-05 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.536347+0000", 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0450000000000006e-06 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.536404+0000", 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7201999999999999e-05 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.536431+0000", 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7495999999999999e-05 2024-09-13T04:58:10.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.537352+0000", 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092032699999999997 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.537391+0000", 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9589000000000003e-05 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.665 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:625 3.1d e29/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 29'10, mlcod=29'8)", 2024-09-13T04:58:10.665 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.539907+0000", 2024-09-13T04:58:10.665 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.1080228569999999, 2024-09-13T04:58:10.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038040719999999999, 2024-09-13T04:58:10.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.539907+0000", 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.539907+0000", 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.539906+0000", 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.539935+0000", 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8583000000000001e-05 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.539947+0000", 2024-09-13T04:58:10.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2235000000000001e-05 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.539952+0000", 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.198e-06 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.540000+0000", 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8066999999999999e-05 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.540023+0000", 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2464999999999999e-05 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.543682+0000", 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036595389999999998 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.543711+0000", 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8719999999999999e-05 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.669 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:667 3.1c e29/24 3:3a486bdd:::rbd_data.112ac9124348.0000000000000013:head v 29'10, mlcod=28'8)", 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.607750+0000", 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.040180004, 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011582470000000001, 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.607750+0000", 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.607750+0000", 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.607748+0000", 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.607816+0000", 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7563000000000003e-05 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.607835+0000", 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9207000000000001e-05 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.607845+0000", 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4960000000000002e-06 2024-09-13T04:58:10.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.607888+0000", 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3455000000000003e-05 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.607906+0000", 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8026e-05 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.608846+0000", 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093979700000000003 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.608908+0000", 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2230999999999994e-05 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:683 3.1d e29/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 29'12, mlcod=29'10)", 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.654969+0000", 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.992960658, 2024-09-13T04:58:10.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003254425, 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.654969+0000", 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.654969+0000", 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.654968+0000", 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.655007+0000", 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8843000000000001e-05 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.655028+0000", 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1664000000000001e-05 2024-09-13T04:58:10.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.655036+0000", 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.655097+0000", 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1591e-05 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.655123+0000", 2024-09-13T04:58:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5646000000000001e-05 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.658183+0000", 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030599720000000002 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.658224+0000", 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0741999999999997e-05 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:688 3.1c e29/24 3:3a486bdd:::rbd_data.112ac9124348.0000000000000013:head v 29'11, mlcod=28'9)", 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.685462+0000", 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96246757299999997, 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00083247300000000002, 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.685462+0000", 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.685462+0000", 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.685461+0000", 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.685473+0000", 2024-09-13T04:58:10.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2519e-05 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.685498+0000", 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.516e-05 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.685513+0000", 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4744e-05 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.685572+0000", 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9011000000000001e-05 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.685597+0000", 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5372e-05 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.686255+0000", 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00065713499999999999 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.686295+0000", 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0339999999999997e-05 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.683 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:693 3.1d e29/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 29'13, mlcod=29'11)", 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.689160+0000", 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.958770236, 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011271569999999999, 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.689160+0000", 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.689160+0000", 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.689159+0000", 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.689166+0000", 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8129999999999998e-06 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.689177+0000", 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1813e-05 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.689185+0000", 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5410000000000003e-06 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.689268+0000", 2024-09-13T04:58:10.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3267000000000001e-05 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.689310+0000", 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1533999999999999e-05 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.690223+0000", 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00091330800000000005 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.690287+0000", 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3721999999999997e-05 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:714 3.1d e29/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 29'14, mlcod=29'12)", 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.728847+0000", 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.91908307899999997, 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037521809999999998, 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.688 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.728847+0000", 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.728847+0000", 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.728845+0000", 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.728877+0000", 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2332999999999997e-05 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.728907+0000", 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0380000000000001e-05 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.728931+0000", 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3765999999999999e-05 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.729029+0000", 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8012999999999998e-05 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.729068+0000", 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8766000000000001e-05 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.732527+0000", 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003458727 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.732599+0000", 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2519000000000001e-05 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:719 3.1d e29/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 29'15, mlcod=29'13)", 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.765767+0000", 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.88216284899999997, 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012002110000000001, 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.765767+0000", 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.765767+0000", 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.765765+0000", 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.765778+0000", 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2802999999999999e-05 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.765804+0000", 2024-09-13T04:58:10.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6010999999999999e-05 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.765816+0000", 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1902999999999999e-05 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.765932+0000", 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011628 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.766002+0000", 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9249000000000001e-05 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.766928+0000", 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092673600000000005 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.766967+0000", 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8859000000000002e-05 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:763 3.1d e29/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 29'16, mlcod=29'14)", 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.830277+0000", 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.81765314499999997, 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002385837, 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830277+0000", 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830277+0000", 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830274+0000", 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999967 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830385+0000", 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011127 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830424+0000", 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8631999999999999e-05 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830442+0000", 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7989999999999999e-05 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830549+0000", 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010713399999999999 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830583+0000", 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4065e-05 2024-09-13T04:58:10.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.832597+0000", 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020146790000000001 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.832663+0000", 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5180999999999998e-05 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:764 3.3 3:c09c3a3e:::rbd_data.112ac9124348.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write 6291456~10568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.830769+0000", 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.81716108200000004, 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025016830000000002, 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 764 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830769+0000", 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830769+0000", 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830766+0000", 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:10.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830798+0000", 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1343999999999997e-05 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830818+0000", 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.056e-05 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830851+0000", 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.269e-05 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830979+0000", 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127441 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.831105+0000", 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126155 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.831342+0000", 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023721199999999999 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.832982+0000", 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0016405090000000001 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.833195+0000", 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021303899999999999 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.833239+0000", 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3339e-05 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.833271+0000", 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1764000000000003e-05 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:782 3.3 3:c4f93521:::rbd_data.112ac9124348.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write 3663130~6886] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:10.708 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.932799+0000", 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.71513095299999996, 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032316630000000001, 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 782 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932799+0000", 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932799+0000", 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932796+0000", 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932820+0000", 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.423e-05 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932869+0000", 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9190000000000002e-05 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932892+0000", 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2730000000000001e-05 2024-09-13T04:58:10.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.933025+0000", 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013330199999999999 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.933206+0000", 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018090399999999999 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:10.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.933528+0000", 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032134999999999998 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.935012+0000", 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014840859999999999 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.935864+0000", 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085221099999999996 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.935938+0000", 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3747999999999999e-05 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.936031+0000", 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2979000000000005e-05 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:810 3.3 3:c09c3a3e:::rbd_data.112ac9124348.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:10.714 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.028070+0000", 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.61985976799999998, 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024610410000000002, 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 810 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028070+0000", 2024-09-13T04:58:10.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028070+0000", 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028067+0000", 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028085+0000", 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7387000000000001e-05 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028130+0000", 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5383000000000002e-05 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028155+0000", 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4698999999999998e-05 2024-09-13T04:58:10.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028244+0000", 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8634999999999994e-05 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028332+0000", 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8573999999999996e-05 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028449+0000", 2024-09-13T04:58:10.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011723899999999999 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.029384+0000", 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093421699999999999 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.030465+0000", 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010816090000000001 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.030500+0000", 2024-09-13T04:58:10.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4530000000000003e-05 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.030531+0000", 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1452000000000003e-05 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:812 3.3 3:c4f93521:::rbd_data.112ac9124348.000000000000000d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.028197+0000", 2024-09-13T04:58:10.720 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.619732913, 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034975200000000001, 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 812 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028197+0000", 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028197+0000", 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028196+0000", 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028204+0000", 2024-09-13T04:58:10.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3340000000000002e-06 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028226+0000", 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2413e-05 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028258+0000", 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2053e-05 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028779+0000", 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00052097999999999999 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028855+0000", 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5492000000000002e-05 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028965+0000", 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109856 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.030429+0000", 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001464506 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.031634+0000", 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012042509999999999 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.031674+0000", 2024-09-13T04:58:10.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0231000000000002e-05 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.031695+0000", 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0803999999999999e-05 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:809 3.1d e29/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 29'17, mlcod=29'15)", 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.028774+0000", 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.61915554299999997, 2024-09-13T04:58:10.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023761780000000001, 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028774+0000", 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028774+0000", 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028773+0000", 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028789+0000", 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6290000000000002e-05 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028817+0000", 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7926e-05 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.028835+0000", 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8090000000000001e-05 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.029006+0000", 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000170922 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.029064+0000", 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8383e-05 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.031044+0000", 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019794740000000002 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.031151+0000", 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010671 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.730 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:991 3.3 3:c4f93521:::rbd_data.112ac9124348.000000000000000d:head [set-alloc-hint object_size 8388608 write_size 8388608,write 8079789~39205] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.540649+0000", 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.10728061699999999, 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047257130000000003, 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 991 2024-09-13T04:58:10.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.540649+0000", 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.540649+0000", 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.540647+0000", 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.540810+0000", 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016291400000000001 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.540840+0000", 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0713999999999998e-05 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.540854+0000", 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4165000000000001e-05 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.540973+0000", 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118959 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.540982+0000", 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8869999999999998e-06 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.543554+0000", 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025712389999999999 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.543609+0000", 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5930000000000002e-05 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.543704+0000", 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4963000000000005e-05 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.544871+0000", 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011663380000000001 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.545314+0000", 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044372999999999998 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.545352+0000", 2024-09-13T04:58:10.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7078e-05 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.545375+0000", 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3499999999999999e-05 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:1022 3.1d e30/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 30'18, mlcod=29'16)", 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.599481+0000", 2024-09-13T04:58:10.737 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.048449352000000001, 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014149849999999999, 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.599481+0000", 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.599481+0000", 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.599480+0000", 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.599523+0000", 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2561999999999997e-05 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.599538+0000", 2024-09-13T04:58:10.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5435999999999999e-05 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.599548+0000", 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7540000000000001e-06 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.599614+0000", 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5693999999999998e-05 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.599640+0000", 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5948999999999998e-05 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.600854+0000", 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012147849999999999 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.600896+0000", 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1156999999999999e-05 2024-09-13T04:58:10.741 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:1030 3.3 3:c4f93521:::rbd_data.112ac9124348.000000000000000d:head [truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.618580+0000", 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.029350444, 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001973991, 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:10.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1030 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618580+0000", 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618580+0000", 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618577+0000", 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618589+0000", 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.154e-05 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618617+0000", 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7678000000000002e-05 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618634+0000", 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7496e-05 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618704+0000", 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0148999999999995e-05 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618791+0000", 2024-09-13T04:58:10.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6712999999999995e-05 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.618920+0000", 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012859199999999999 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619864+0000", 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094435500000000002 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:10.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.620464+0000", 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00059983900000000001 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.620515+0000", 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0881000000000003e-05 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.620553+0000", 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8791999999999997e-05 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:1029 3.1d e30/24 3:bd9359fa:::rbd_data.112ac9124348.0000000000000009:head v 30'19, mlcod=29'17)", 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.619251+0000", 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.028679050000000001, 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00091150599999999997, 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619251+0000", 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619251+0000", 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619250+0000", 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619259+0000", 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0110000000000006e-06 2024-09-13T04:58:10.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619272+0000", 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3013999999999999e-05 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619279+0000", 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7999999999999999e-06 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:10.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619364+0000", 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5000999999999995e-05 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.619387+0000", 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2472000000000001e-05 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.620119+0000", 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073192500000000002 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:10.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:10.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.620162+0000", 2024-09-13T04:58:10.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3551999999999999e-05 2024-09-13T04:58:10.752 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.752 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.752 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.752 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:10.752 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:10.752 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:10.753 INFO:teuthology.orchestra.run.smithi001.stdout:320 writesame 0xaa4eebd thru 0xaa54916 data_size 0x2d(0x5a5a bytes) 2024-09-13T04:58:10.753 INFO:teuthology.orchestra.run.smithi001.stdout:321 writesame 0xcfbdd68 thru 0xcfc3f2a data_size 0x1(0x61c3 bytes) 2024-09-13T04:58:10.753 INFO:teuthology.orchestra.run.smithi001.stdout:325 read 0x9a15e6b thru 0x9a259e0 (0xfb76 bytes) 2024-09-13T04:58:10.753 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.754 INFO:teuthology.orchestra.run.smithi001.stdout:327 punch from 0x501cd04 to 0x5027308, (0xa604 bytes) 2024-09-13T04:58:10.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:328 punch from 0xa326ddb to 0xa32f963, (0x8b88 bytes) 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:330 writesame 0x23e62ce thru 0x23e9d11 data_size 0x42(0x3a44 bytes) 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:331 writesame 0xa577bef thru 0xa5809c0 data_size 0x9(0x8dd2 bytes) 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:333 trunc from 0xcfc3f2b to 0x934e848 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:334 write 0x2964abf thru 0x296f9cb (0xaf0d bytes) 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:335 read 0x7e49294 thru 0x7e4da53 (0x47c0 bytes) 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:337 trunc from 0x934e848 to 0xc7cba8d 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:340 trunc from 0xc7cba8d to 0x293585f 2024-09-13T04:58:10.756 INFO:teuthology.orchestra.run.smithi001.stdout:341 read 0x349977 thru 0x35213f (0x87c9 bytes) 2024-09-13T04:58:10.757 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:10.752+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:10.786 INFO:teuthology.orchestra.run.smithi001.stdout:342 write 0xd87a627 thru 0xd87f2ce (0x4ca8 bytes) 2024-09-13T04:58:10.793 INFO:teuthology.orchestra.run.smithi001.stdout:343 punch from 0x1c90fa6 to 0x1c9428b, (0x32e5 bytes) 2024-09-13T04:58:10.793 INFO:teuthology.orchestra.run.smithi001.stdout:344 read 0x69c8d05 thru 0x69d47a0 (0xba9c bytes) 2024-09-13T04:58:10.793 INFO:teuthology.orchestra.run.smithi001.stdout:345 punch from 0x4e4de91 to 0x4e5c421, (0xe590 bytes) 2024-09-13T04:58:10.793 INFO:teuthology.orchestra.run.smithi001.stdout:349 write 0x27bf3a0 thru 0x27c2170 (0x2dd1 bytes) 2024-09-13T04:58:10.796 INFO:teuthology.orchestra.run.smithi001.stdout:350 punch from 0xd4bb892 to 0xd4c28f8, (0x7066 bytes) 2024-09-13T04:58:10.796 INFO:teuthology.orchestra.run.smithi001.stdout:352 write 0x5a30ada thru 0x5a3e257 (0xd77e bytes) 2024-09-13T04:58:10.805 INFO:teuthology.orchestra.run.smithi001.stdout:353 read 0x4e068aa thru 0x4e12d19 (0xc470 bytes) 2024-09-13T04:58:10.805 INFO:teuthology.orchestra.run.smithi001.stdout:355 writesame 0xbaccc8f thru 0xbacdb5a data_size 0x2(0xecc bytes) 2024-09-13T04:58:10.825 INFO:teuthology.orchestra.run.smithi001.stdout:356 trunc from 0xd87f2cf to 0x6d3d50b 2024-09-13T04:58:10.842 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xe32211f 2024-09-13T04:58:10.855 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:10.852+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:10.855 INFO:teuthology.orchestra.run.smithi001.stdout:358 trunc from 0x6d3d50b to 0xe32211f 2024-09-13T04:58:10.863 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.863 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.863 INFO:teuthology.orchestra.run.smithi001.stdout:362 punch from 0x9b0a5c3 to 0x9b179aa, (0xd3e7 bytes) 2024-09-13T04:58:10.863 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.863 INFO:teuthology.orchestra.run.smithi001.stdout:364 read 0x6450278 thru 0x645a252 (0x9fdb bytes) 2024-09-13T04:58:10.863 INFO:teuthology.orchestra.run.smithi001.stdout:365 writesame 0x90f03ee thru 0x90f7246 data_size 0x29(0x6e59 bytes) 2024-09-13T04:58:10.868 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.868 INFO:teuthology.orchestra.run.smithi001.stdout:369 writesame 0xd13df7e thru 0xd149f30 data_size 0x41(0xbfb3 bytes) 2024-09-13T04:58:10.875 INFO:teuthology.orchestra.run.smithi001.stdout:371 writesame 0xca1c74b thru 0xca28fe7 data_size 0x39(0xc89d bytes) 2024-09-13T04:58:10.879 INFO:teuthology.orchestra.run.smithi001.stdout:372 punch from 0x5e0086b to 0x5e02d91, (0x2526 bytes) 2024-09-13T04:58:10.879 INFO:teuthology.orchestra.run.smithi001.stdout:373 writesame 0x6c5ab8f thru 0x6c6404b data_size 0x1d(0x94bd bytes) 2024-09-13T04:58:10.887 INFO:teuthology.orchestra.run.smithi001.stdout:374 trunc from 0xe32211f to 0xcc3bbdb 2024-09-13T04:58:10.898 INFO:teuthology.orchestra.run.smithi001.stdout:376 read 0x644e11f thru 0x6453fa2 (0x5e84 bytes) 2024-09-13T04:58:10.898 INFO:teuthology.orchestra.run.smithi001.stdout:377 read 0x9421889 thru 0x9422269 (0x9e1 bytes) 2024-09-13T04:58:10.898 INFO:teuthology.orchestra.run.smithi001.stdout:378 read 0x5cd1931 thru 0x5ce00e0 (0xe7b0 bytes) 2024-09-13T04:58:10.898 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.898 INFO:teuthology.orchestra.run.smithi001.stdout:381 punch from 0x2b68c41 to 0x2b6df6b, (0x532a bytes) 2024-09-13T04:58:10.898 INFO:teuthology.orchestra.run.smithi001.stdout:382 read 0x2ea43b4 thru 0x2eacda2 (0x89ef bytes) 2024-09-13T04:58:10.899 INFO:teuthology.orchestra.run.smithi001.stdout:383 writesame 0x86929ef thru 0x86992a7 data_size 0x13(0x68b9 bytes) 2024-09-13T04:58:10.907 INFO:teuthology.orchestra.run.smithi001.stdout:384 punch from 0xa143a9c to 0xa1450fb, (0x165f bytes) 2024-09-13T04:58:10.907 INFO:teuthology.orchestra.run.smithi001.stdout:386 write 0xac35cb thru 0xac37fc (0x232 bytes) 2024-09-13T04:58:10.910 INFO:teuthology.orchestra.run.smithi001.stdout:387 read 0xa5dd83b thru 0xa5ecc83 (0xf449 bytes) 2024-09-13T04:58:10.911 INFO:teuthology.orchestra.run.smithi001.stdout:388 write 0x376fa78 thru 0x37761be (0x6747 bytes) 2024-09-13T04:58:10.916 INFO:teuthology.orchestra.run.smithi001.stdout:389 punch from 0x3f6bb5e to 0x3f7a4df, (0xe981 bytes) 2024-09-13T04:58:10.916 INFO:teuthology.orchestra.run.smithi001.stdout:392 punch from 0x9d6280f to 0x9d71584, (0xed75 bytes) 2024-09-13T04:58:10.916 INFO:teuthology.orchestra.run.smithi001.stdout:394 punch from 0x8bdedd4 to 0x8be47c9, (0x59f5 bytes) 2024-09-13T04:58:10.917 INFO:teuthology.orchestra.run.smithi001.stdout:395 trunc from 0xcc3bbdb to 0xbf6985e 2024-09-13T04:58:10.929 INFO:teuthology.orchestra.run.smithi001.stdout:396 write 0xb5ec1e8 thru 0xb5f80cc (0xbee5 bytes) 2024-09-13T04:58:10.936 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.936 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:10.939 INFO:teuthology.orchestra.run.smithi001.stdout:400 trunc from 0xbf6985e to 0x980e41d 2024-09-13T04:58:10.950 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:10.950 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:10.950 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:10.950 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:10.955 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:10.952+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:10.956 INFO:teuthology.orchestra.run.smithi001.stdout:401 write 0x1949e1b thru 0x194d868 (0x3a4e bytes) 2024-09-13T04:58:10.960 INFO:teuthology.orchestra.run.smithi001.stdout:403 write 0x85b208c thru 0x85b6d4f (0x4cc4 bytes) 2024-09-13T04:58:10.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:10.976 INFO:teuthology.orchestra.run.smithi001.stdout:404 write 0xa31d1da thru 0xa31d7ca (0x5f1 bytes) 2024-09-13T04:58:10.991 INFO:teuthology.orchestra.run.smithi001.stdout:406 trunc from 0xa31d7cb to 0x4023f14 2024-09-13T04:58:11.007 INFO:teuthology.orchestra.run.smithi001.stdout:407 read 0x10d2afb thru 0x10d7457 (0x495d bytes) 2024-09-13T04:58:11.007 INFO:teuthology.orchestra.run.smithi001.stdout:408 punch from 0x1cb05d3 to 0x1cb6f39, (0x6966 bytes) 2024-09-13T04:58:11.019 INFO:teuthology.orchestra.run.smithi001.stdout:409 write 0x6a10eb6 thru 0x6a15d46 (0x4e91 bytes) 2024-09-13T04:58:11.023 INFO:teuthology.orchestra.run.smithi001.stdout:411 read 0x474d22f thru 0x4755df4 (0x8bc6 bytes) 2024-09-13T04:58:11.025 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.034 INFO:teuthology.orchestra.run.smithi001.stdout:413 trunc from 0x6a15d47 to 0xca08c1c 2024-09-13T04:58:11.041 INFO:teuthology.orchestra.run.smithi001.stdout:414 punch from 0x752c27 to 0x755303, (0x26dc bytes) 2024-09-13T04:58:11.042 INFO:teuthology.orchestra.run.smithi001.stdout:417 punch from 0x39d9849 to 0x39e6aaa, (0xd261 bytes) 2024-09-13T04:58:11.042 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.042 INFO:teuthology.orchestra.run.smithi001.stdout:421 write 0x218df43 thru 0x218ec74 (0xd32 bytes) 2024-09-13T04:58:11.045 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.045 INFO:teuthology.orchestra.run.smithi001.stdout:425 read 0xb2f7bbc thru 0xb305032 (0xd477 bytes) 2024-09-13T04:58:11.045 INFO:teuthology.orchestra.run.smithi001.stdout:428 write 0xf2e09b thru 0xf3d237 (0xf19d bytes) 2024-09-13T04:58:11.054 INFO:teuthology.orchestra.run.smithi001.stdout:430 write 0xb50e56c thru 0xb51366c (0x5101 bytes) 2024-09-13T04:58:11.056 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:11.052+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:11.062 INFO:teuthology.orchestra.run.smithi001.stdout:431 writesame 0x19e7c3d thru 0x19f0d7c data_size 0x53(0x9140 bytes) 2024-09-13T04:58:11.066 INFO:teuthology.orchestra.run.smithi001.stdout:432 writesame 0x46851eb thru 0x468d02c data_size 0x3(0x7e42 bytes) 2024-09-13T04:58:11.081 INFO:teuthology.orchestra.run.smithi001.stdout:433 write 0x8f449cf thru 0x8f48ea3 (0x44d5 bytes) 2024-09-13T04:58:11.087 INFO:teuthology.orchestra.run.smithi001.stdout:434 punch from 0x6714b98 to 0x6722d85, (0xe1ed bytes) 2024-09-13T04:58:11.087 INFO:teuthology.orchestra.run.smithi001.stdout:436 read 0x2eb61b8 thru 0x2ec5e06 (0xfc4f bytes) 2024-09-13T04:58:11.088 INFO:teuthology.orchestra.run.smithi001.stdout:437 clone 2 order 22 su 1048576 sc 13 2024-09-13T04:58:11.118 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:11.118 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:11.118 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:11.132 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:11.177 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:11.176+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:11.248 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T04:58:11.258 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:11.256+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:11.271 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:58:11.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T04:58:11.315 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.188465+0000", 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.125177623999999, 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01139475, 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 36 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188465+0000", 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188465+0000", 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188464+0000", 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188471+0000", 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188480+0000", 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.234e-06 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188488+0000", 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2460000000000003e-06 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.195918+0000", 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074296099999999997 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.196162+0000", 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024367799999999999 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.196551+0000", 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038946 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.199645+0000", 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030943490000000001 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.199666+0000", 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0414000000000001e-05 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.199796+0000", 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013059200000000001 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.199859+0000", 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2836999999999995e-05 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.188497+0000", 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.125145407000002, 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011748669999999999, 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 37 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188497+0000", 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188497+0000", 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188496+0000", 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188503+0000", 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5599999999999999e-06 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188512+0000", 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2399999999999996e-06 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.188521+0000", 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7059999999999993e-06 2024-09-13T04:58:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.198071+0000", 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095504070000000003 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.198349+0000", 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027834399999999998 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.198570+0000", 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022022900000000001 2024-09-13T04:58:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.199877+0000", 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013074 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.200110+0000", 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023295100000000001 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.200188+0000", 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7608999999999995e-05 2024-09-13T04:58:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.200245+0000", 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7883999999999997e-05 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.444692+0000", 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868949883999999, 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011575821, 2024-09-13T04:58:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 109 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444692+0000", 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444692+0000", 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444691+0000", 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444791+0000", 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010001300000000001 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444805+0000", 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3484e-05 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444814+0000", 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8489999999999995e-06 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.446960+0000", 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021461610000000002 2024-09-13T04:58:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.447033+0000", 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3342000000000004e-05 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.447126+0000", 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3413999999999997e-05 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450034+0000", 2024-09-13T04:58:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029071790000000002 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456122+0000", 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006088052 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456213+0000", 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1508999999999994e-05 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456268+0000", 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4870000000000002e-05 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.444826+0000", 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868816506999998, 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011570387, 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 110 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444826+0000", 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444826+0000", 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444825+0000", 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444832+0000", 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0230000000000004e-06 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444841+0000", 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4409999999999999e-06 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444848+0000", 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9720000000000003e-06 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.448190+0000", 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033420949999999998 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.448250+0000", 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9558000000000001e-05 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.448343+0000", 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2780000000000003e-05 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450693+0000", 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023504799999999998 2024-09-13T04:58:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456313+0000", 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056198799999999998 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456355+0000", 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1482e-05 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456396+0000", 2024-09-13T04:58:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1488e-05 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.444873+0000", 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868769458999999, 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011731816000000001, 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 111 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444873+0000", 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444873+0000", 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444872+0000", 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444879+0000", 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0339999999999999e-06 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444889+0000", 2024-09-13T04:58:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0467e-05 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444907+0000", 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7640000000000001e-05 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.448684+0000", 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037770170000000001 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.448741+0000", 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7095000000000001e-05 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.448830+0000", 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8676000000000003e-05 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450698+0000", 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001868291 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456473+0000", 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057752869999999996 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456548+0000", 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4302999999999997e-05 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456605+0000", 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6818999999999999e-05 2024-09-13T04:58:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.444939+0000", 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868702971000001, 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011802003, 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 112 2024-09-13T04:58:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444939+0000", 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444939+0000", 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.444938+0000", 2024-09-13T04:58:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445039+0000", 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101078 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445053+0000", 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3234000000000001e-05 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445066+0000", 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3682999999999999e-05 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.449313+0000", 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042467269999999996 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.449372+0000", 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8836999999999999e-05 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.449450+0000", 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7662000000000003e-05 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.452450+0000", 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030000280000000001 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456662+0000", 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042127739999999999 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456696+0000", 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3760999999999997e-05 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456741+0000", 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5008e-05 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.445079+0000", 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868563192, 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01186072, 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445079+0000", 2024-09-13T04:58:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445079+0000", 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445078+0000", 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445084+0000", 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2870000000000001e-06 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445094+0000", 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0329999999999997e-06 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445102+0000", 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0989999999999999e-06 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450067+0000", 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049652580000000002 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450153+0000", 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6507000000000004e-05 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450260+0000", 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106833 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.454068+0000", 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038079569999999998 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456811+0000", 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002742782 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456889+0000", 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7966000000000005e-05 2024-09-13T04:58:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456940+0000", 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.445192+0000", 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868450126999999, 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011880392, 2024-09-13T04:58:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 114 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445192+0000", 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445192+0000", 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445191+0000", 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445199+0000", 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4290000000000004e-06 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445239+0000", 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0000000000000003e-05 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445248+0000", 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4599999999999992e-06 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450732+0000", 2024-09-13T04:58:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054835099999999996 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450820+0000", 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8676000000000003e-05 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.450915+0000", 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4636999999999995e-05 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.454089+0000", 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003173726 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.456995+0000", 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029062789999999999 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457027+0000", 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2604000000000002e-05 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457072+0000", 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4938999999999998e-05 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.445403+0000", 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868239065000001, 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011860206999999999, 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 115 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445403+0000", 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445403+0000", 2024-09-13T04:58:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445402+0000", 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445506+0000", 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104143 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445522+0000", 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6300999999999999e-05 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445531+0000", 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3000000000000002e-06 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.451251+0000", 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057203169999999999 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.451303+0000", 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2151999999999998e-05 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.451413+0000", 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109261 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.454102+0000", 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026891129999999999 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457136+0000", 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030344740000000001 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457207+0000", 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.059e-05 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457263+0000", 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6619000000000001e-05 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.445541+0000", 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868101283000001, 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01185105, 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 116 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445541+0000", 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445541+0000", 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445540+0000", 2024-09-13T04:58:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445546+0000", 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.485e-06 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445554+0000", 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3890000000000005e-06 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445562+0000", 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8830000000000006e-06 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.452502+0000", 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069402960000000003 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.452560+0000", 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8150999999999999e-05 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.452653+0000", 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2974999999999997e-05 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.454113+0000", 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001459286 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457316+0000", 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032031899999999999 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457348+0000", 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1816999999999997e-05 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457392+0000", 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4360999999999998e-05 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.445571+0000", 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.868071603000001, 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012001563, 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 117 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445571+0000", 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445571+0000", 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445570+0000", 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445575+0000", 2024-09-13T04:58:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0440000000000003e-06 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445585+0000", 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4870000000000008e-06 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.445602+0000", 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7856e-05 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.453164+0000", 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075614890000000002 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.453310+0000", 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146606 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.453577+0000", 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026632799999999997 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.454569+0000", 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00099215900000000005 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457451+0000", 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028824950000000001 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457522+0000", 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0079000000000004e-05 2024-09-13T04:58:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.457572+0000", 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0569999999999999e-05 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.938235+0000", 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.37540753, 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011587794, 2024-09-13T04:58:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 133 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938235+0000", 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938235+0000", 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938233+0000", 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938246+0000", 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.26e-05 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938263+0000", 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7203000000000001e-05 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938272+0000", 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3330000000000003e-06 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.944593+0000", 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006320432 2024-09-13T04:58:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.944837+0000", 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024465799999999997 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.945162+0000", 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032485800000000003 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.948570+0000", 2024-09-13T04:58:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003407513 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.949650+0000", 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001079932 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.949721+0000", 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1335999999999996e-05 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.949822+0000", 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010133599999999999 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.938449+0000", 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.375193239000001, 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01264384, 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 134 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938449+0000", 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938449+0000", 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938448+0000", 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938601+0000", 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015288199999999999 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938627+0000", 2024-09-13T04:58:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.654e-05 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938644+0000", 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6713000000000001e-05 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.946097+0000", 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074535000000000001 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.946279+0000", 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018195799999999999 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.946569+0000", 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028918200000000002 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.949972+0000", 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003403246 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.950925+0000", 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00095347599999999998 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.950993+0000", 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8085999999999998e-05 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.951093+0000", 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9439999999999997e-05 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.938667+0000", 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.374975556999999, 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015300159000000001, 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 135 2024-09-13T04:58:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938667+0000", 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938667+0000", 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938666+0000", 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938677+0000", 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1399999999999999e-05 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938689+0000", 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2016e-05 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938696+0000", 2024-09-13T04:58:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4370000000000001e-06 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.948643+0000", 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0099469970000000008 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.948774+0000", 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013063899999999999 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.948997+0000", 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022302800000000001 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.953657+0000", 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046600310000000002 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.953779+0000", 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122009 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.953856+0000", 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6792999999999995e-05 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.953967+0000", 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110889 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.938705+0000", 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.374937317000001, 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021159629999999999, 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 136 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938705+0000", 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938705+0000", 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938704+0000", 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938710+0000", 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938719+0000", 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2930000000000006e-06 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938726+0000", 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1400000000000002e-06 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.950027+0000", 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01130038 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.950146+0000", 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119392 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.950372+0000", 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022560500000000001 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.953760+0000", 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033884679999999999 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.959620+0000", 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005859698 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.959757+0000", 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013677900000000001 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.959864+0000", 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107762 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.938734+0000", 2024-09-13T04:58:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.374907710999999, 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021644947000000001, 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 137 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938734+0000", 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938734+0000", 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938734+0000", 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938921+0000", 2024-09-13T04:58:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018729200000000001 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938939+0000", 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7706999999999998e-05 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938948+0000", 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7030000000000001e-06 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.951211+0000", 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012262732 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.951469+0000", 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000257905 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.951817+0000", 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034776000000000002 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.959976+0000", 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.008159164 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960145+0000", 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016906899999999999 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960277+0000", 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013179199999999999 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960379+0000", 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102513 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.938962+0000", 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.374680193, 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021651940000000001, 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 138 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938962+0000", 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938962+0000", 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938961+0000", 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938974+0000", 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3117e-05 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938984+0000", 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.011e-05 2024-09-13T04:58:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938991+0000", 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1099999999999997e-06 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.954054+0000", 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015062837000000001 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.954179+0000", 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012523600000000001 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.954465+0000", 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028531800000000001 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960120+0000", 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005654999 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960404+0000", 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028399499999999999 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960522+0000", 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117822 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960614+0000", 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2349000000000006e-05 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.938999+0000", 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.374642846, 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021931257999999999, 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 139 2024-09-13T04:58:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938999+0000", 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938999+0000", 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.938999+0000", 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939009+0000", 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0793e-05 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939019+0000", 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0087e-05 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939027+0000", 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0729999999999999e-06 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.955086+0000", 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016059276000000001 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.955204+0000", 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118313 2024-09-13T04:58:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.955412+0000", 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020786200000000001 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960638+0000", 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052259120000000001 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960747+0000", 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010946900000000001 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960823+0000", 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5473000000000001e-05 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960931+0000", 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107723 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.939040+0000", 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.374602110000001, 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022206858999999999, 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 140 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939040+0000", 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939040+0000", 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939039+0000", 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939192+0000", 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152239 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939206+0000", 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4066000000000001e-05 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939214+0000", 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4270000000000008e-06 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.956127+0000", 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016912522999999999 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.956245+0000", 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118379 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.956480+0000", 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000234868 2024-09-13T04:58:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.960655+0000", 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041750149999999998 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.961039+0000", 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000384037 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.961156+0000", 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116845 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.961247+0000", 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1143000000000005e-05 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4163.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T04:58:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:50.939225+0000", 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.374416858, 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022271888, 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4163", 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3941865217", 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 141 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939225+0000", 2024-09-13T04:58:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939225+0000", 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939224+0000", 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939235+0000", 2024-09-13T04:58:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0603e-05 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939245+0000", 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5429999999999999e-06 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.939252+0000", 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9e-06 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.958064+0000", 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018812265000000002 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.958324+0000", 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000260015 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.958681+0000", 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00035768399999999999 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.961276+0000", 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025945030000000002 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.961362+0000", 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5649000000000006e-05 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.961443+0000", 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1126999999999998e-05 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:50.961497+0000", 2024-09-13T04:58:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4432000000000003e-05 2024-09-13T04:58:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:11.431 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:11.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:11.434 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone1 from 0xca08c1c (overlap 0x9927b2) to 0x0 2024-09-13T04:58:11.434 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-13T04:58:11.434 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:11.432+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:11.533 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:11.532+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:11.599 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:11.602 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:11.602 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:11.602 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:11.602 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:11.619 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:11.661 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:11.660+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:11.735 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:11.736+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:11.741 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:11.746 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.746 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.746 INFO:teuthology.orchestra.run.smithi001.stdout:441 write 0x41d7b30 thru 0x41dacaf (0x3180 bytes) 2024-09-13T04:58:11.759 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:11.759 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:11.760 INFO:teuthology.orchestra.run.smithi001.stdout:442 writesame 0x5779d5f thru 0x578440d data_size 0x1(0xa6af bytes) 2024-09-13T04:58:11.786 INFO:teuthology.orchestra.run.smithi001.stdout:443 write 0x36b53b4 thru 0x36c5198 (0xfde5 bytes) 2024-09-13T04:58:11.791 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:11.791 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:11.800 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:11.801 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:11.817 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:11.817 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:11.817 INFO:teuthology.orchestra.run.smithi001.stdout:445 trunc from 0xca08c1c to 0x17f013f 2024-09-13T04:58:11.828 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:11.836 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:11.836+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:11.844 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:11.870 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:11.870 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:11.871 INFO:teuthology.orchestra.run.smithi001.stdout:446 read 0x79f3a7 thru 0x7a8d97 (0x99f1 bytes) 2024-09-13T04:58:11.872 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.886 INFO:teuthology.orchestra.run.smithi001.stdout:449 write 0x5e3085d thru 0x5e38484 (0x7c28 bytes) 2024-09-13T04:58:11.894 INFO:teuthology.orchestra.run.smithi001.stdout:450 punch from 0x132b33 to 0x13a632, (0x7aff bytes) 2024-09-13T04:58:11.894 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.897 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:11.897 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:11.905 INFO:teuthology.orchestra.run.smithi001.stdout:453 trunc from 0x5e38485 to 0xd5c8a7a 2024-09-13T04:58:11.912 INFO:teuthology.orchestra.run.smithi001.stdout:454 write 0xbc88e4a thru 0xbc8c921 (0x3ad8 bytes) 2024-09-13T04:58:11.917 INFO:teuthology.orchestra.run.smithi001.stdout:455 read 0x20df5d9 thru 0x20ef421 (0xfe49 bytes) 2024-09-13T04:58:11.918 INFO:teuthology.orchestra.run.smithi001.stdout:458 punch from 0x455bcf to 0x4563fc, (0x82d bytes) 2024-09-13T04:58:11.918 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.918 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.918 INFO:teuthology.orchestra.run.smithi001.stdout:464 write 0x69abb1b thru 0x69b07a3 (0x4c89 bytes) 2024-09-13T04:58:11.922 INFO:teuthology.orchestra.run.smithi001.stdout:465 read 0xb269e79 thru 0xb26d71d (0x38a5 bytes) 2024-09-13T04:58:11.922 INFO:teuthology.orchestra.run.smithi001.stdout:466 writesame 0x2ddc9d thru 0x2e9759 data_size 0x5(0xbabd bytes) 2024-09-13T04:58:11.925 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:11.925 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:11.937 INFO:teuthology.orchestra.run.smithi001.stdout:467 trunc from 0xd5c8a7a to 0x941fd83 2024-09-13T04:58:11.937 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:11.936+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:11.952 INFO:teuthology.orchestra.run.smithi001.stdout:471 read 0x89465ba thru 0x894ed57 (0x879e bytes) 2024-09-13T04:58:11.952 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.962 INFO:teuthology.orchestra.run.smithi001.stdout:474 writesame 0xb830167 thru 0xb83c86d data_size 0x1(0xc707 bytes) 2024-09-13T04:58:11.962 INFO:teuthology.orchestra.run.smithi001.stdout:475 write 0x49c9dc6 thru 0x49cf97d (0x5bb8 bytes) 2024-09-13T04:58:11.968 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:11.968 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:11.969 INFO:teuthology.orchestra.run.smithi001.stdout:476 writesame 0xa791d66 thru 0xa799a4e data_size 0xb(0x7ce9 bytes) 2024-09-13T04:58:11.976 INFO:teuthology.orchestra.run.smithi001.stdout:477 punch from 0x9e2775b to 0x9e2cb08, (0x53ad bytes) 2024-09-13T04:58:11.976 INFO:teuthology.orchestra.run.smithi001.stdout:478 punch from 0x9153816 to 0x9155796, (0x1f80 bytes) 2024-09-13T04:58:11.976 INFO:teuthology.orchestra.run.smithi001.stdout:480 punch from 0x3ce096 to 0x3dc683, (0xe5ed bytes) 2024-09-13T04:58:11.976 INFO:teuthology.orchestra.run.smithi001.stdout:484 punch from 0x47aed81 to 0x47bd753, (0xe9d2 bytes) 2024-09-13T04:58:11.976 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:11.976 INFO:teuthology.orchestra.run.smithi001.stdout:486 read 0x15d580 thru 0x165576 (0x7ff7 bytes) 2024-09-13T04:58:11.985 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:11.986 INFO:teuthology.orchestra.run.smithi001.stdout:489 trunc from 0xb83c86e to 0x484c4bd 2024-09-13T04:58:12.003 INFO:teuthology.orchestra.run.smithi001.stdout:492 punch from 0x14c3820 to 0x14cad8c, (0x756c bytes) 2024-09-13T04:58:12.003 INFO:teuthology.orchestra.run.smithi001.stdout:494 punch from 0x28974a to 0x290699, (0x6f4f bytes) 2024-09-13T04:58:12.021 INFO:teuthology.orchestra.run.smithi001.stdout:495 write 0x9fbc0be thru 0x9fc17dc (0x571f bytes) 2024-09-13T04:58:12.023 INFO:teuthology.orchestra.run.smithi001.stdout:497 writesame 0x6b94834 thru 0x6ba2ec9 data_size 0x5(0xe696 bytes) 2024-09-13T04:58:12.035 INFO:teuthology.orchestra.run.smithi001.stdout:500 read 0x6cb5849 thru 0x6cbb5a3 (0x5d5b bytes) 2024-09-13T04:58:12.035 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.035 INFO:teuthology.orchestra.run.smithi001.stdout:502 read 0x68f1079 thru 0x68ffc7f (0xec07 bytes) 2024-09-13T04:58:12.035 INFO:teuthology.orchestra.run.smithi001.stdout:503 writesame 0x41f0e7d thru 0x41fd144 data_size 0x8(0xc2c8 bytes) 2024-09-13T04:58:12.038 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:12.036+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:12.045 INFO:teuthology.orchestra.run.smithi001.stdout:504 read 0x29df609 thru 0x29eaf64 (0xb95c bytes) 2024-09-13T04:58:12.045 INFO:teuthology.orchestra.run.smithi001.stdout:505 read 0x91d5233 thru 0x91d8b9e (0x396c bytes) 2024-09-13T04:58:12.046 INFO:teuthology.orchestra.run.smithi001.stdout:506 write 0x8440808 thru 0x844455b (0x3d54 bytes) 2024-09-13T04:58:12.050 INFO:teuthology.orchestra.run.smithi001.stdout:507 read 0x1b9d93c thru 0x1ba1ffc (0x46c1 bytes) 2024-09-13T04:58:12.051 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.051 INFO:teuthology.orchestra.run.smithi001.stdout:509 punch from 0x5c1d3c0 to 0x5c2710f, (0x9d4f bytes) 2024-09-13T04:58:12.051 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.051 INFO:teuthology.orchestra.run.smithi001.stdout:511 writesame 0x2113c7c thru 0x2123686 data_size 0x13(0xfa0b bytes) 2024-09-13T04:58:12.056 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.056 INFO:teuthology.orchestra.run.smithi001.stdout:515 write 0x335acce thru 0x336246a (0x779d bytes) 2024-09-13T04:58:12.062 INFO:teuthology.orchestra.run.smithi001.stdout:516 read 0x3847029 thru 0x3856657 (0xf62f bytes) 2024-09-13T04:58:12.062 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.065 INFO:teuthology.orchestra.run.smithi001.stdout:519 trunc from 0x9fc17dd to 0xbd6fe78 2024-09-13T04:58:12.071 INFO:teuthology.orchestra.run.smithi001.stdout:520 read 0x29d4423 thru 0x29d9fec (0x5bca bytes) 2024-09-13T04:58:12.071 INFO:teuthology.orchestra.run.smithi001.stdout:521 read 0x2caea38 thru 0x2cbd9d7 (0xefa0 bytes) 2024-09-13T04:58:12.072 INFO:teuthology.orchestra.run.smithi001.stdout:522 write 0x1b40e7d thru 0x1b4a695 (0x9819 bytes) 2024-09-13T04:58:12.079 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.079 INFO:teuthology.orchestra.run.smithi001.stdout:524 punch from 0x7b23f04 to 0x7b2623f, (0x233b bytes) 2024-09-13T04:58:12.079 INFO:teuthology.orchestra.run.smithi001.stdout:525 write 0x85fbb6c thru 0x860458a (0x8a1f bytes) 2024-09-13T04:58:12.086 INFO:teuthology.orchestra.run.smithi001.stdout:526 punch from 0x6e1c6e2 to 0x6e256e6, (0x9004 bytes) 2024-09-13T04:58:12.098 INFO:teuthology.orchestra.run.smithi001.stdout:528 trunc from 0xbd6fe78 to 0x3b4d2a3 2024-09-13T04:58:12.115 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:12.115 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:12.139 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:12.136+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:12.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:12.147 INFO:teuthology.orchestra.run.smithi001.stdout:529 write 0xdfd19c6 thru 0xdfd717d (0x57b8 bytes) 2024-09-13T04:58:12.153 INFO:teuthology.orchestra.run.smithi001.stdout:533 writesame 0x7d50c4e thru 0x7d52d4d data_size 0x18(0x2100 bytes) 2024-09-13T04:58:12.160 INFO:teuthology.orchestra.run.smithi001.stdout:534 write 0x86c78c2 thru 0x86cac21 (0x3360 bytes) 2024-09-13T04:58:12.165 INFO:teuthology.orchestra.run.smithi001.stdout:535 read 0x8c33b21 thru 0x8c33c75 (0x155 bytes) 2024-09-13T04:58:12.165 INFO:teuthology.orchestra.run.smithi001.stdout:536 read 0x7d3fd7d thru 0x7d40ee4 (0x1168 bytes) 2024-09-13T04:58:12.166 INFO:teuthology.orchestra.run.smithi001.stdout:537 read 0x17b9212 thru 0x17c1ff2 (0x8de1 bytes) 2024-09-13T04:58:12.167 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.167 INFO:teuthology.orchestra.run.smithi001.stdout:539 write 0xa3af30 thru 0xa3d6a0 (0x2771 bytes) 2024-09-13T04:58:12.171 INFO:teuthology.orchestra.run.smithi001.stdout:540 punch from 0x3f8558 to 0x403f7b, (0xba23 bytes) 2024-09-13T04:58:12.171 INFO:teuthology.orchestra.run.smithi001.stdout:542 punch from 0xd95f2e2 to 0xd96701e, (0x7d3c bytes) 2024-09-13T04:58:12.171 INFO:teuthology.orchestra.run.smithi001.stdout:545 write 0xdf28d2e thru 0xdf2cfa2 (0x4275 bytes) 2024-09-13T04:58:12.174 INFO:teuthology.orchestra.run.smithi001.stdout:546 write 0x61eb108 thru 0x61ec1a4 (0x109d bytes) 2024-09-13T04:58:12.188 INFO:teuthology.orchestra.run.smithi001.stdout:547 writesame 0xe1898ba thru 0xe192854 data_size 0x1(0x8f9b bytes) 2024-09-13T04:58:12.188 INFO:teuthology.orchestra.run.smithi001.stdout:549 punch from 0xd7c0ee8 to 0xd7cfec6, (0xefde bytes) 2024-09-13T04:58:12.188 INFO:teuthology.orchestra.run.smithi001.stdout:550 read 0x30319e2 thru 0x303988f (0x7eae bytes) 2024-09-13T04:58:12.194 INFO:teuthology.orchestra.run.smithi001.stdout:551 trunc from 0xe192855 to 0xa87323f 2024-09-13T04:58:12.222 INFO:teuthology.orchestra.run.smithi001.stdout:552 write 0xe61473f thru 0xe623304 (0xebc6 bytes) 2024-09-13T04:58:12.229 INFO:teuthology.orchestra.run.smithi001.stdout:553 write 0x4dbb207 thru 0x4dbf8f1 (0x46eb bytes) 2024-09-13T04:58:12.235 INFO:teuthology.orchestra.run.smithi001.stdout:554 write 0x37f49a6 thru 0x37f7a6d (0x30c8 bytes) 2024-09-13T04:58:12.239 INFO:teuthology.orchestra.run.smithi001.stdout:555 write 0x82b2d8b thru 0x82c0d68 (0xdfde bytes) 2024-09-13T04:58:12.239 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:12.236+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:12.247 INFO:teuthology.orchestra.run.smithi001.stdout:556 read 0x8860d99 thru 0x8861459 (0x6c1 bytes) 2024-09-13T04:58:12.247 INFO:teuthology.orchestra.run.smithi001.stdout:557 write 0xbe127f5 thru 0xbe22029 (0xf835 bytes) 2024-09-13T04:58:12.261 INFO:teuthology.orchestra.run.smithi001.stdout:558 punch from 0xd6dc227 to 0xd6e61da, (0x9fb3 bytes) 2024-09-13T04:58:12.261 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.261 INFO:teuthology.orchestra.run.smithi001.stdout:560 writesame 0x19d0bd8 thru 0x19ddd3c data_size 0x47(0xd165 bytes) 2024-09-13T04:58:12.264 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T04:58:12.264 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:58:12.266 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.266 INFO:teuthology.orchestra.run.smithi001.stdout:563 read 0xbc66d3d thru 0xbc7517f (0xe443 bytes) 2024-09-13T04:58:12.267 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.267 INFO:teuthology.orchestra.run.smithi001.stdout:570 write 0x57718a1 thru 0x578168b (0xfdeb bytes) 2024-09-13T04:58:12.280 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:12.280 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:12.282 INFO:teuthology.orchestra.run.smithi001.stdout:571 punch from 0xda9e0cb to 0xdaa334b, (0x5280 bytes) 2024-09-13T04:58:12.282 INFO:teuthology.orchestra.run.smithi001.stdout:572 read 0x1851ca9 thru 0x1859543 (0x789b bytes) 2024-09-13T04:58:12.285 INFO:teuthology.orchestra.run.smithi001.stdout:573 read 0x8563ea9 thru 0x8568460 (0x45b8 bytes) 2024-09-13T04:58:12.285 INFO:teuthology.orchestra.run.smithi001.stdout:574 writesame 0xd973b47 thru 0xd97a061 data_size 0xd(0x651b bytes) 2024-09-13T04:58:12.287 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T04:58:12.295 INFO:teuthology.orchestra.run.smithi001.stdout:575 punch from 0xab9f6ee to 0xabab4db, (0xbded bytes) 2024-09-13T04:58:12.295 INFO:teuthology.orchestra.run.smithi001.stdout:576 write 0x3097826 thru 0x309b256 (0x3a31 bytes) 2024-09-13T04:58:12.297 INFO:teuthology.orchestra.run.smithi001.stdout:578 read 0x1ef07fa thru 0x1efec6f (0xe476 bytes) 2024-09-13T04:58:12.297 INFO:teuthology.orchestra.run.smithi001.stdout:580 writesame 0x9fdb2c1 thru 0x9fe9e6f data_size 0xb(0xebaf bytes) 2024-09-13T04:58:12.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:585 writesame 0x6df89c7 thru 0x6e01dd0 data_size 0x7(0x940a bytes) 2024-09-13T04:58:12.334 INFO:teuthology.orchestra.run.smithi001.stdout:586 trunc from 0xe623305 to 0x2334d59 2024-09-13T04:58:12.341 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:12.340+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:12.365 INFO:teuthology.orchestra.run.smithi001.stdout:587 read 0x16a33cc thru 0x16a8a10 (0x5645 bytes) 2024-09-13T04:58:12.389 INFO:teuthology.orchestra.run.smithi001.stdout:588 write 0xd933f96 thru 0xd93a1f3 (0x625e bytes) 2024-09-13T04:58:12.395 INFO:teuthology.orchestra.run.smithi001.stdout:589 read 0x89e43eb thru 0x89ef4d3 (0xb0e9 bytes) 2024-09-13T04:58:12.405 INFO:teuthology.orchestra.run.smithi001.stdout:590 trunc from 0xd93a1f4 to 0x64a63aa 2024-09-13T04:58:12.422 INFO:teuthology.orchestra.run.smithi001.stdout:591 write 0xf33610 thru 0xf41746 (0xe137 bytes) 2024-09-13T04:58:12.436 INFO:teuthology.orchestra.run.smithi001.stdout:592 writesame 0x6f4d99a thru 0x6f5d7a4 data_size 0x1(0xfe0b bytes) 2024-09-13T04:58:12.441 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:12.440+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:12.444 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:12.444 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:12.448 INFO:teuthology.orchestra.run.smithi001.stdout:594 write 0xb019ffe thru 0xb026a89 (0xca8c bytes) 2024-09-13T04:58:12.453 INFO:teuthology.orchestra.run.smithi001.stdout:595 read 0x79b8b40 thru 0x79c3eb2 (0xb373 bytes) 2024-09-13T04:58:12.466 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:12.467 INFO:teuthology.orchestra.run.smithi001.stdout:596 trunc from 0xb026a8a to 0x1b4514c 2024-09-13T04:58:12.491 INFO:teuthology.orchestra.run.smithi001.stdout:600 write 0x236e8d2 thru 0x237e34b (0xfa7a bytes) 2024-09-13T04:58:12.513 INFO:teuthology.orchestra.run.smithi001.stdout:602 writesame 0xc4dc7e0 thru 0xc4e5487 data_size 0x1c(0x8ca8 bytes) 2024-09-13T04:58:12.514 INFO:teuthology.orchestra.run.smithi001.stdout:603 read 0x2460e8e thru 0x2461739 (0x8ac bytes) 2024-09-13T04:58:12.531 INFO:teuthology.orchestra.run.smithi001.stdout:604 trunc from 0xc4e5488 to 0x154e2ff 2024-09-13T04:58:12.542 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:12.540+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:12.545 INFO:teuthology.orchestra.run.smithi001.stdout:605 read 0xccef3d thru 0xcd5c70 (0x6d34 bytes) 2024-09-13T04:58:12.571 INFO:teuthology.orchestra.run.smithi001.stdout:606 writesame 0xd58372d thru 0xd58740e data_size 0x1(0x3ce2 bytes) 2024-09-13T04:58:12.571 INFO:teuthology.orchestra.run.smithi001.stdout:608 write 0x7c01c7c thru 0x7c03a44 (0x1dc9 bytes) 2024-09-13T04:58:12.575 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.575 INFO:teuthology.orchestra.run.smithi001.stdout:612 write 0xc926f9b thru 0xc930ee2 (0x9f48 bytes) 2024-09-13T04:58:12.581 INFO:teuthology.orchestra.run.smithi001.stdout:613 write 0x4730f19 thru 0x473938f (0x8477 bytes) 2024-09-13T04:58:12.591 INFO:teuthology.orchestra.run.smithi001.stdout:615 punch from 0xa1a1ab to 0xa1b3de, (0x1233 bytes) 2024-09-13T04:58:12.594 INFO:teuthology.orchestra.run.smithi001.stdout:616 trunc from 0xd58740f to 0xaccd6db 2024-09-13T04:58:12.600 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:12.601 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:12.606 INFO:teuthology.orchestra.run.smithi001.stdout:617 read 0x99d887f thru 0x99e2286 (0x9a08 bytes) 2024-09-13T04:58:12.606 INFO:teuthology.orchestra.run.smithi001.stdout:620 writesame 0x5d35ed0 thru 0x5d3fcd0 data_size 0x61(0x9e01 bytes) 2024-09-13T04:58:12.614 INFO:teuthology.orchestra.run.smithi001.stdout:621 write 0x725be91 thru 0x7264c8d (0x8dfd bytes) 2024-09-13T04:58:12.615 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:12.621 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.622 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.622 INFO:teuthology.orchestra.run.smithi001.stdout:625 read 0x687c231 thru 0x687f943 (0x3713 bytes) 2024-09-13T04:58:12.622 INFO:teuthology.orchestra.run.smithi001.stdout:627 trunc from 0xaccd6db to 0xac6cdb5 2024-09-13T04:58:12.628 INFO:teuthology.orchestra.run.smithi001.stdout:628 write 0x90e5c55 thru 0x90f1b9f (0xbf4b bytes) 2024-09-13T04:58:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:629 punch from 0x551e to 0xd4b8, (0x7f9a bytes) 2024-09-13T04:58:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:630 punch from 0x5655472 to 0x56643b9, (0xef47 bytes) 2024-09-13T04:58:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:12.639 INFO:teuthology.orchestra.run.smithi001.stdout:634 read 0x61a906b thru 0x61aff8e (0x6f24 bytes) 2024-09-13T04:58:12.640 INFO:teuthology.orchestra.run.smithi001.stdout:635 punch from 0x7ca70f0 to 0x7cb4d2b, (0xdc3b bytes) 2024-09-13T04:58:12.640 INFO:teuthology.orchestra.run.smithi001.stdout:636 punch from 0x93b115 to 0x949be7, (0xead2 bytes) 2024-09-13T04:58:12.640 INFO:teuthology.orchestra.run.smithi001.stdout:637 clone 3 order 19 su 4096 sc 10 2024-09-13T04:58:12.642 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:12.640+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:12.737 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:12.737 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:12.743 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:12.740+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:12.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:12.845 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:12.844+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:12.910 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:12.911 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:12.925 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:12.972 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:12.972+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:13.047 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:13.044+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:13.049 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:13.050 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:13.064 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:13.147 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:13.144+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:13.232 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:13.232 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:13.246 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:13.247 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:13.244+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:13.272 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T04:58:13.302 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T04:58:13.349 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:13.348+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:13.367 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:13.367 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:13.382 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:13.384 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone2 from 0xac6cdb5 (overlap 0x154e2ff) to 0x2216b3 2024-09-13T04:58:13.449 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:13.448+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:13.491 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-13T04:58:13.492 INFO:teuthology.orchestra.run.smithi001.stdout:638 punch from 0x8c867c5 to 0x8c8b5ff, (0x4e3a bytes) 2024-09-13T04:58:13.497 INFO:teuthology.orchestra.run.smithi001.stdout:639 read 0x59873db thru 0x598c252 (0x4e78 bytes) 2024-09-13T04:58:13.498 INFO:teuthology.orchestra.run.smithi001.stdout:640 trunc from 0xac6cdb5 to 0xa86b57b 2024-09-13T04:58:13.514 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:13.514 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:13.528 INFO:teuthology.orchestra.run.smithi001.stdout:642 write 0xb23db83 thru 0xb244527 (0x69a5 bytes) 2024-09-13T04:58:13.530 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:13.540 INFO:teuthology.orchestra.run.smithi001.stdout:646 trunc from 0xb244528 to 0xbc00bb1 2024-09-13T04:58:13.549 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:13.548+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:13.554 INFO:teuthology.orchestra.run.smithi001.stdout:647 write 0xbc33961 thru 0xbc37d71 (0x4411 bytes) 2024-09-13T04:58:13.563 INFO:teuthology.orchestra.run.smithi001.stdout:648 punch from 0xaa7c707 to 0xaa87e80, (0xb779 bytes) 2024-09-13T04:58:13.563 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:13.563 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xec9061d 2024-09-13T04:58:13.568 INFO:teuthology.orchestra.run.smithi001.stdout:651 trunc from 0xbc37d72 to 0xec9061d 2024-09-13T04:58:13.577 INFO:teuthology.orchestra.run.smithi001.stdout:652 read 0x324c0e7 thru 0x324c9a0 (0x8ba bytes) 2024-09-13T04:58:13.578 INFO:teuthology.orchestra.run.smithi001.stdout:653 punch from 0xcabd218 to 0xcac1c5d, (0x4a45 bytes) 2024-09-13T04:58:13.578 INFO:teuthology.orchestra.run.smithi001.stdout:657 read 0x40099c2 thru 0x4018a2d (0xf06c bytes) 2024-09-13T04:58:13.578 INFO:teuthology.orchestra.run.smithi001.stdout:658 punch from 0x2b07650 to 0x2b0a2ac, (0x2c5c bytes) 2024-09-13T04:58:13.578 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:13.578 INFO:teuthology.orchestra.run.smithi001.stdout:661 write 0x820b262 thru 0x8210ff4 (0x5d93 bytes) 2024-09-13T04:58:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:662 read 0x2a0aa3d thru 0x2a125c7 (0x7b8b bytes) 2024-09-13T04:58:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:665 read 0xe30296d thru 0xe30893d (0x5fd1 bytes) 2024-09-13T04:58:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:666 writesame 0x4d72ac4 thru 0x4d80de1 data_size 0x7(0xe31e bytes) 2024-09-13T04:58:13.622 INFO:teuthology.orchestra.run.smithi001.stdout:667 writesame 0x2ec12be thru 0x2ec2dfe data_size 0x1(0x1b41 bytes) 2024-09-13T04:58:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:668 write 0x30e881a thru 0x30f43c2 (0xbba9 bytes) 2024-09-13T04:58:13.650 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:13.648+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:13.720 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:13.720 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:13.724 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:13.724 INFO:teuthology.orchestra.run.smithi001.stdout:671 read 0x58e11f1 thru 0x58e8d71 (0x7b81 bytes) 2024-09-13T04:58:13.725 INFO:teuthology.orchestra.run.smithi001.stdout:672 write 0x6f43a8c thru 0x6f4fc2a (0xc19f bytes) 2024-09-13T04:58:13.735 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:13.752 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:13.748+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:13.789 INFO:teuthology.orchestra.run.smithi001.stdout:674 read 0x2b1c3a6 thru 0x2b24514 (0x816f bytes) 2024-09-13T04:58:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:676 read 0x528c27e thru 0x5296186 (0x9f09 bytes) 2024-09-13T04:58:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:677 write 0xd030903 thru 0xd031b53 (0x1251 bytes) 2024-09-13T04:58:13.794 INFO:teuthology.orchestra.run.smithi001.stdout:678 writesame 0x1d92f49 thru 0x1d9ab4c data_size 0x2(0x7c04 bytes) 2024-09-13T04:58:13.852 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:13.852+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:13.864 INFO:teuthology.orchestra.run.smithi001.stdout:679 trunc from 0xec9061d to 0x6340794 2024-09-13T04:58:13.872 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:13.872 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:13.892 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:13.905 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T04:58:13.928 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T04:58:13.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:13.969 INFO:teuthology.orchestra.run.smithi001.stdout:681 punch from 0x276c3b3 to 0x276f4ac, (0x30f9 bytes) 2024-09-13T04:58:13.969 INFO:teuthology.orchestra.run.smithi001.stdout:682 writesame 0x519370 thru 0x51a5f8 data_size 0xd(0x1289 bytes) 2024-09-13T04:58:13.979 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:13.980+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:13.996 INFO:teuthology.orchestra.run.smithi001.stdout:683 punch from 0x2e78fc7 to 0x2e7f289, (0x62c2 bytes) 2024-09-13T04:58:14.010 INFO:teuthology.orchestra.run.smithi001.stdout:685 write 0xa7f776f thru 0xa7fc860 (0x50f2 bytes) 2024-09-13T04:58:14.024 INFO:teuthology.orchestra.run.smithi001.stdout:686 punch from 0x76cb3f8 to 0x76d0e32, (0x5a3a bytes) 2024-09-13T04:58:14.033 INFO:teuthology.orchestra.run.smithi001.stdout:687 write 0xa8a69ca thru 0xa8ae155 (0x778c bytes) 2024-09-13T04:58:14.044 INFO:teuthology.orchestra.run.smithi001.stdout:688 read 0x5939001 thru 0x593b9bc (0x29bc bytes) 2024-09-13T04:58:14.044 INFO:teuthology.orchestra.run.smithi001.stdout:691 read 0x2f9f19e thru 0x2fa9521 (0xa384 bytes) 2024-09-13T04:58:14.045 INFO:teuthology.orchestra.run.smithi001.stdout:692 read 0x4e9809c thru 0x4e986c8 (0x62d bytes) 2024-09-13T04:58:14.046 INFO:teuthology.orchestra.run.smithi001.stdout:694 read 0x9ef009a thru 0x9ef90bc (0x9023 bytes) 2024-09-13T04:58:14.046 INFO:teuthology.orchestra.run.smithi001.stdout:696 read 0x7b4106 thru 0x7b4310 (0x20b bytes) 2024-09-13T04:58:14.046 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:14.054 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:14.052+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:14.059 INFO:teuthology.orchestra.run.smithi001.stdout:698 writesame 0xe076ee8 thru 0xe0865aa data_size 0x3(0xf6c3 bytes) 2024-09-13T04:58:14.069 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:14.069 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:14.071 INFO:teuthology.orchestra.run.smithi001.stdout:699 trunc from 0xe0865ab to 0x4eb0459 2024-09-13T04:58:14.082 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:14.097 INFO:teuthology.orchestra.run.smithi001.stdout:700 write 0x1669329 thru 0x166bcc4 (0x299c bytes) 2024-09-13T04:58:14.127 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:14.139 INFO:teuthology.orchestra.run.smithi001.stdout:701 write 0x97491c1 thru 0x974e13f (0x4f7f bytes) 2024-09-13T04:58:14.145 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:14.145 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:14.154 INFO:teuthology.orchestra.run.smithi001.stdout:702 write 0xa337d51 thru 0xa3475dc (0xf88c bytes) 2024-09-13T04:58:14.155 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:14.152+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:14.167 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:14.169 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:14.179 INFO:teuthology.orchestra.run.smithi001.stdout:704 write 0xe6a8aa6 thru 0xe6b6c6d (0xe1c8 bytes) 2024-09-13T04:58:14.185 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:14.191 INFO:teuthology.orchestra.run.smithi001.stdout:705 write 0xd6674c6 thru 0xd67391c (0xc457 bytes) 2024-09-13T04:58:14.202 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:14.202 INFO:teuthology.orchestra.run.smithi001.stdout:710 write 0x68f963b thru 0x68fc2e3 (0x2ca9 bytes) 2024-09-13T04:58:14.208 INFO:teuthology.orchestra.run.smithi001.stdout:712 read 0x57fda51 thru 0x580749d (0x9a4d bytes) 2024-09-13T04:58:14.208 INFO:teuthology.orchestra.run.smithi001.stdout:713 punch from 0x1f77314 to 0x1f77da3, (0xa8f bytes) 2024-09-13T04:58:14.208 INFO:teuthology.orchestra.run.smithi001.stdout:714 write 0x14b41cf thru 0x14c36d5 (0xf507 bytes) 2024-09-13T04:58:14.213 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:14.213 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:14.239 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:14.240 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:14.256 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:14.256+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:14.256 INFO:teuthology.orchestra.run.smithi001.stdout:715 write 0x4c03e32 thru 0x4c04ee1 (0x10b0 bytes) 2024-09-13T04:58:14.262 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:14.262 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:14.267 INFO:teuthology.orchestra.run.smithi001.stdout:716 trunc from 0xe6b6c6e to 0x7363a51 2024-09-13T04:58:14.273 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:14.273 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:14.287 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:14.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:58:14.290 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:14.291 INFO:teuthology.orchestra.run.smithi001.stdout:719 punch from 0x1032402 to 0x1038580, (0x617e bytes) 2024-09-13T04:58:14.304 INFO:teuthology.orchestra.run.smithi001.stdout:720 writesame 0xcd4631d thru 0xcd51402 data_size 0x1(0xb0e6 bytes) 2024-09-13T04:58:14.304 INFO:teuthology.orchestra.run.smithi001.stdout:721 write 0x96de4cc thru 0x96e79c4 (0x94f9 bytes) 2024-09-13T04:58:14.324 INFO:teuthology.orchestra.run.smithi001.stdout:723 writesame 0xd8f0b6a thru 0xd8feed7 data_size 0x2b(0xe36e bytes) 2024-09-13T04:58:14.324 INFO:teuthology.orchestra.run.smithi001.stdout:724 write 0xbbb9250 thru 0xbbb9bf4 (0x9a5 bytes) 2024-09-13T04:58:14.327 INFO:teuthology.orchestra.run.smithi001.stdout:727 write 0xd4a03d2 thru 0xd4a640d (0x603c bytes) 2024-09-13T04:58:14.335 INFO:teuthology.orchestra.run.smithi001.stdout:728 write 0x8fd6ffa thru 0x8fe3ef7 (0xcefe bytes) 2024-09-13T04:58:14.349 INFO:teuthology.orchestra.run.smithi001.stdout:729 read 0x84659e1 thru 0x8473257 (0xd877 bytes) 2024-09-13T04:58:14.349 INFO:teuthology.orchestra.run.smithi001.stdout:730 writesame 0x18090b4 thru 0x18173ea data_size 0x45(0xe337 bytes) 2024-09-13T04:58:14.356 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:14.352+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:14.362 INFO:teuthology.orchestra.run.smithi001.stdout:734 write 0xdf77aa6 thru 0xdf806a3 (0x8bfe bytes) 2024-09-13T04:58:14.376 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:14.376 INFO:teuthology.orchestra.run.smithi001.stdout:737 punch from 0xd8d3642 to 0xd8d589c, (0x225a bytes) 2024-09-13T04:58:14.377 INFO:teuthology.orchestra.run.smithi001.stdout:738 punch from 0x11db868 to 0x11dca3f, (0x11d7 bytes) 2024-09-13T04:58:14.377 INFO:teuthology.orchestra.run.smithi001.stdout:739 read 0x6e1850d thru 0x6e25753 (0xd247 bytes) 2024-09-13T04:58:14.377 INFO:teuthology.orchestra.run.smithi001.stdout:740 punch from 0x24968d to 0x249d56, (0x6c9 bytes) 2024-09-13T04:58:14.377 INFO:teuthology.orchestra.run.smithi001.stdout:742 trunc from 0xdf806a4 to 0xd21cd3d 2024-09-13T04:58:14.393 INFO:teuthology.orchestra.run.smithi001.stdout:743 read 0x26db739 thru 0x26eb452 (0xfd1a bytes) 2024-09-13T04:58:14.396 INFO:teuthology.orchestra.run.smithi001.stdout:748 write 0x4091bc8 thru 0x409e937 (0xcd70 bytes) 2024-09-13T04:58:14.408 INFO:teuthology.orchestra.run.smithi001.stdout:749 write 0x61d1e40 thru 0x61d87ba (0x697b bytes) 2024-09-13T04:58:14.420 INFO:teuthology.orchestra.run.smithi001.stdout:750 clone 4 order 25 su 1048576 sc 6 2024-09-13T04:58:14.458 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:14.456+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7735 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'193, mlcod=34'188)", 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.309054+0000", 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.14995619700000001, 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046153649999999997, 2024-09-13T04:58:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309054+0000", 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309054+0000", 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309053+0000", 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309059+0000", 2024-09-13T04:58:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.604e-06 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309067+0000", 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4970000000000001e-06 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309071+0000", 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1280000000000001e-06 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309095+0000", 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3371999999999999e-05 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309110+0000", 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5577999999999999e-05 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.313636+0000", 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045258900000000003 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.313669+0000", 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2978000000000002e-05 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7736 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'194, mlcod=34'190)", 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.309689+0000", 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.14932103899999999, 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0040066429999999998, 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309689+0000", 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309689+0000", 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309688+0000", 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309692+0000", 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8510000000000001e-06 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309699+0000", 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4520000000000002e-06 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309702+0000", 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9340000000000002e-06 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309761+0000", 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9197000000000003e-05 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.309775+0000", 2024-09-13T04:58:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4426e-05 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.313671+0000", 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0038953770000000002 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.313695+0000", 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4822000000000001e-05 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7737 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'195, mlcod=34'190)", 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.310197+0000", 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.14881330200000001, 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035251240000000001, 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310197+0000", 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310197+0000", 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310196+0000", 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310200+0000", 2024-09-13T04:58:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.613e-06 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310206+0000", 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.19e-06 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310211+0000", 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7879999999999997e-06 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310235+0000", 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3876999999999999e-05 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310251+0000", 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5999999999999999e-05 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.313698+0000", 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034473780000000001 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.313722+0000", 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3621999999999998e-05 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7738 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'196, mlcod=34'190)", 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.310947+0000", 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.148062995, 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0055694810000000003, 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310947+0000", 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310947+0000", 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310946+0000", 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310952+0000", 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.246e-06 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310965+0000", 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2962000000000001e-05 2024-09-13T04:58:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.310972+0000", 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2750000000000001e-06 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311011+0000", 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8942000000000001e-05 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311027+0000", 2024-09-13T04:58:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6733000000000001e-05 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.316482+0000", 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0054543669999999999 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.316516+0000", 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4613000000000002e-05 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7739 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'197, mlcod=34'190)", 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.311761+0000", 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.147249093, 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0047803189999999999, 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311761+0000", 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311761+0000", 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311760+0000", 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311765+0000", 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1390000000000003e-06 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311776+0000", 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0574000000000001e-05 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311780+0000", 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.605e-06 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311816+0000", 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5710000000000002e-05 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.311836+0000", 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9724e-05 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.316519+0000", 2024-09-13T04:58:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046832829999999999 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.316541+0000", 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1849e-05 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7740 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'198, mlcod=34'191)", 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.312405+0000", 2024-09-13T04:58:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.14660452600000001, 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0041476409999999997, 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312405+0000", 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312405+0000", 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312405+0000", 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312408+0000", 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.388e-06 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312417+0000", 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1380000000000007e-06 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312420+0000", 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2399999999999999e-06 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312456+0000", 2024-09-13T04:58:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6371000000000001e-05 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312470+0000", 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3862e-05 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.316542+0000", 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0040720349999999999 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.316553+0000", 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0892e-05 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7741 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'199, mlcod=34'191)", 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.312954+0000", 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.14605574800000001, 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036113909999999998, 2024-09-13T04:58:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312954+0000", 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312954+0000", 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312954+0000", 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312960+0000", 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6799999999999998e-06 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312967+0000", 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1609999999999997e-06 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.312971+0000", 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1869999999999999e-06 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.313003+0000", 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2391000000000002e-05 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.313016+0000", 2024-09-13T04:58:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2625e-05 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.316554+0000", 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035383799999999998 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.316565+0000", 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1212999999999999e-05 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7784 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'217, mlcod=34'213)", 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.343404+0000", 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.11560598699999999, 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049813560000000002, 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.343404+0000", 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.343404+0000", 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.343403+0000", 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.343409+0000", 2024-09-13T04:58:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9769999999999997e-06 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.343421+0000", 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2037e-05 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.343426+0000", 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1730000000000003e-06 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.343462+0000", 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5880999999999997e-05 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.343476+0000", 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4007e-05 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.348363+0000", 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0048871330000000001 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.348385+0000", 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1866999999999999e-05 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7791 3.8 e34/24 3:121cb06c:::rbd_data.112a2c87bb71.000000000000011c:head v 34'19, mlcod=34'17)", 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.344130+0000", 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.11488000399999999, 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043322710000000004, 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344130+0000", 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344130+0000", 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344129+0000", 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344153+0000", 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4147000000000001e-05 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344165+0000", 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1609e-05 2024-09-13T04:58:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344170+0000", 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3360000000000002e-06 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344220+0000", 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0142999999999997e-05 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344243+0000", 2024-09-13T04:58:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2676000000000001e-05 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.348423+0000", 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0041803209999999999 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.348462+0000", 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8761999999999999e-05 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7785 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'218, mlcod=34'214)", 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.344287+0000", 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.114722897, 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0041346919999999997, 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344287+0000", 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344287+0000", 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344286+0000", 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344295+0000", 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4719999999999995e-06 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344305+0000", 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9359999999999994e-06 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344316+0000", 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1685000000000001e-05 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344356+0000", 2024-09-13T04:58:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9665000000000001e-05 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344375+0000", 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8941999999999999e-05 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.348387+0000", 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0040124109999999996 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.348422+0000", 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4323999999999999e-05 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7786 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'219, mlcod=34'214)", 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.344923+0000", 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.11408642500000001, 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035243150000000001, 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344923+0000", 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344923+0000", 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344923+0000", 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344937+0000", 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3732e-05 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344945+0000", 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5499999999999995e-06 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344949+0000", 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8970000000000001e-06 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344979+0000", 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9754999999999999e-05 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.344991+0000", 2024-09-13T04:58:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1983e-05 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.348424+0000", 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0034330929999999999 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.348448+0000", 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3825e-05 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7822 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'228, mlcod=34'225)", 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.368885+0000", 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.090124877000000006, 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046296250000000001, 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.368885+0000", 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.368885+0000", 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.368884+0000", 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.368901+0000", 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6319000000000001e-05 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.368915+0000", 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195e-05 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.368922+0000", 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.593e-06 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.368969+0000", 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6761000000000002e-05 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.368987+0000", 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8232000000000001e-05 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.373461+0000", 2024-09-13T04:58:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0044740190000000001 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.373515+0000", 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3204000000000001e-05 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7823 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'229, mlcod=34'225)", 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.369821+0000", 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.089188787000000005, 2024-09-13T04:58:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037214290000000001, 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.369821+0000", 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.369821+0000", 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.369820+0000", 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.369828+0000", 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9049999999999997e-06 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.369848+0000", 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0000000000000002e-05 2024-09-13T04:58:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.369856+0000", 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4649999999999997e-06 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.369904+0000", 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8757000000000001e-05 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.369927+0000", 2024-09-13T04:58:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2850999999999999e-05 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.373518+0000", 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035905580000000002 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.373542+0000", 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4723999999999999e-05 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7824 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'230, mlcod=34'225)", 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.370711+0000", 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.088298689999999999, 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0054497920000000002, 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.370711+0000", 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.370711+0000", 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.370710+0000", 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.370717+0000", 2024-09-13T04:58:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0339999999999999e-06 2024-09-13T04:58:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.370731+0000", 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3546e-05 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.370738+0000", 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8360000000000003e-06 2024-09-13T04:58:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.370772+0000", 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4347999999999997e-05 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.370785+0000", 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2999e-05 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.376124+0000", 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053393160000000002 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.376161+0000", 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6572999999999997e-05 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7825 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'231, mlcod=34'225)", 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.371224+0000", 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.087785966000000007, 2024-09-13T04:58:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049605780000000002, 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371224+0000", 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371224+0000", 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371223+0000", 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371227+0000", 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7510000000000002e-06 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371238+0000", 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0771000000000001e-05 2024-09-13T04:58:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371244+0000", 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.925e-06 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371278+0000", 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4329000000000003e-05 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371291+0000", 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2446000000000001e-05 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.376163+0000", 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0048725569999999996 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.376184+0000", 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1310000000000001e-05 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7826 3.c e34/24 3:30f391d2:::rbd_data.112a2c87bb71.00000000000001b8:head v 34'232, mlcod=34'226)", 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.371856+0000", 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.087154160999999994, 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043532279999999998, 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371856+0000", 2024-09-13T04:58:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371856+0000", 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371855+0000", 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371863+0000", 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0930000000000003e-06 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371872+0000", 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8249999999999994e-06 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371878+0000", 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7230000000000001e-06 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371911+0000", 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3135e-05 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.371923+0000", 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1588e-05 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.376187+0000", 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0042644420000000002 2024-09-13T04:58:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.376209+0000", 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1942000000000001e-05 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7837 3.c e34/24 3:30f391d2:::rbd_data.112a2c87bb71.00000000000001b8:head v 34'234, mlcod=34'232)", 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.381814+0000", 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.077195546000000004, 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035792340000000001, 2024-09-13T04:58:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.381814+0000", 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.381814+0000", 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.381814+0000", 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.381822+0000", 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.473e-06 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.381842+0000", 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9570999999999999e-05 2024-09-13T04:58:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.381861+0000", 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9006999999999999e-05 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.382491+0000", 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00063051800000000005 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.382512+0000", 2024-09-13T04:58:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1325e-05 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.385364+0000", 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028510580000000001 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.385394+0000", 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9998e-05 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7902 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'249, mlcod=34'246)", 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.411100+0000", 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.047909748000000002, 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037178390000000001, 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.411100+0000", 2024-09-13T04:58:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.411100+0000", 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.411100+0000", 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.411106+0000", 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.973e-06 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.411116+0000", 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2900000000000008e-06 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.411121+0000", 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8300000000000003e-06 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.411160+0000", 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9851000000000002e-05 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.411173+0000", 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2576e-05 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.414779+0000", 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036058589999999999 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.414818+0000", 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9026999999999998e-05 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7907 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'254, mlcod=34'247)", 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.414915+0000", 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.044094793, 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0055061490000000001, 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.414915+0000", 2024-09-13T04:58:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.414915+0000", 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.414914+0000", 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.414929+0000", 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5140000000000001e-05 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.414970+0000", 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0445000000000003e-05 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.414989+0000", 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9602e-05 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415043+0000", 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3897999999999998e-05 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415064+0000", 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.048e-05 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.420388+0000", 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0053240980000000002 2024-09-13T04:58:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.420421+0000", 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3586e-05 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:7908 3.c e34/24 3:32021523:::rbd_object_map.112a2c87bb71:head v 34'255, mlcod=34'247)", 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:14.415803+0000", 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.043206899999999999, 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0046423539999999996, 2024-09-13T04:58:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415803+0000", 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415803+0000", 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415802+0000", 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415808+0000", 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2190000000000003e-06 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415818+0000", 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0249e-05 2024-09-13T04:58:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415823+0000", 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8300000000000003e-06 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415862+0000", 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8825000000000002e-05 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.415881+0000", 2024-09-13T04:58:14.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9009e-05 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.420424+0000", 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045437200000000002 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:14.420445+0000", 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.101e-05 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:14.554 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:14.554 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:14.555 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:14.596 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:14.596+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:14.660 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:14.660+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:14.673 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:14.674 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:14.674 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:14.674 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:14.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:14.760 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:14.760+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:14.807 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:14.808 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:14.808 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:14.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:14.861 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:14.860+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:14.962 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:14.960+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:14.984 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:6692 3.c 3:32021523:::rbd_object_map.112a2c87bb71:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.156808+0000", 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.82760168999999995, 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012624916, 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:14.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6692 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156808+0000", 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156808+0000", 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:14.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156807+0000", 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156811+0000", 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.241e-06 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156817+0000", 2024-09-13T04:58:14.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6900000000000003e-06 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156822+0000", 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9989999999999999e-06 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.163495+0000", 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066731619999999998 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.163578+0000", 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2632000000000004e-05 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.163816+0000", 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000238345 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.164706+0000", 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00088982199999999997 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.169340+0000", 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046339800000000002 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.169390+0000", 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0154999999999997e-05 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.169433+0000", 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2339999999999998e-05 2024-09-13T04:58:14.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:6693 3.c 3:32021523:::rbd_object_map.112a2c87bb71:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.156875+0000", 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.82753422300000001, 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012685834999999999, 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:14.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 6693 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156875+0000", 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156875+0000", 2024-09-13T04:58:14.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156875+0000", 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156879+0000", 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6090000000000003e-06 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156888+0000", 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7600000000000008e-06 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.156895+0000", 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.187e-06 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.164851+0000", 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079558809999999997 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.164915+0000", 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3958e-05 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.165195+0000", 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028047599999999999 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.166255+0000", 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010592889999999999 2024-09-13T04:58:14.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.169475+0000", 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032205860000000001 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.169524+0000", 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9107000000000003e-05 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.169561+0000", 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6584000000000001e-05 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7643 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1548288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224702+0000", 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.759706989, 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011603061, 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:14.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7643 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224702+0000", 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224702+0000", 2024-09-13T04:58:14.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224702+0000", 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224705+0000", 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999999e-06 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:14.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224711+0000", 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4799999999999998e-06 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224716+0000", 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-06 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236234+0000", 2024-09-13T04:58:15.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01151805 2024-09-13T04:58:15.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236268+0000", 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4428999999999999e-05 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236306+0000", 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7515999999999998e-05 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7644 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1589248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224719+0000", 2024-09-13T04:58:15.002 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75969004399999995, 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011685216, 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7644 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224719+0000", 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224719+0000", 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224719+0000", 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224721+0000", 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.289e-06 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224727+0000", 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.733e-06 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224732+0000", 2024-09-13T04:58:15.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6600000000000003e-06 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236320+0000", 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01158786 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236373+0000", 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3309000000000001e-05 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236405+0000", 2024-09-13T04:58:15.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1676e-05 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7645 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224736+0000", 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.759673394, 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011745475999999999, 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7645 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224736+0000", 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224736+0000", 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224736+0000", 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224738+0000", 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7880000000000001e-06 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224743+0000", 2024-09-13T04:58:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224747+0000", 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3379999999999998e-06 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236417+0000", 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011670043 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236452+0000", 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4946000000000001e-05 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236482+0000", 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9329999999999999e-05 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7646 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224751+0000", 2024-09-13T04:58:15.011 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75965846000000004, 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011820410999999999, 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7646 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224751+0000", 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224751+0000", 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224751+0000", 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224753+0000", 2024-09-13T04:58:15.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1179999999999999e-06 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224759+0000", 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0109999999999997e-06 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224764+0000", 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3430000000000003e-06 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236499+0000", 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011734789000000001 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236541+0000", 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1692e-05 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236571+0000", 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0765000000000001e-05 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7647 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1712128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224769+0000", 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75964083800000004, 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011879249999999999, 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7647 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.016 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224769+0000", 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224769+0000", 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224768+0000", 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224771+0000", 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5770000000000001e-06 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224777+0000", 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224781+0000", 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4429999999999996e-06 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236585+0000", 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011803258 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236619+0000", 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4717e-05 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236648+0000", 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.862e-05 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.021 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7648 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.021 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224786+0000", 2024-09-13T04:58:15.021 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75962362900000002, 2024-09-13T04:58:15.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011949352999999999, 2024-09-13T04:58:15.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7648 2024-09-13T04:58:15.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.023 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224786+0000", 2024-09-13T04:58:15.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224786+0000", 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224785+0000", 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224787+0000", 2024-09-13T04:58:15.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1330000000000002e-06 2024-09-13T04:58:15.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224793+0000", 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7309999999999998e-06 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224798+0000", 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3059999999999997e-06 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236660+0000", 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01186163 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236699+0000", 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8528000000000001e-05 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236735+0000", 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6534e-05 2024-09-13T04:58:15.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7649 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1794048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224803+0000", 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75960651999999995, 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012028186999999999, 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7649 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224803+0000", 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224803+0000", 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224802+0000", 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224805+0000", 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4260000000000002e-06 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224811+0000", 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8490000000000002e-06 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224815+0000", 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5700000000000003e-06 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236748+0000", 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011932486000000001 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236784+0000", 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5766999999999998e-05 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236831+0000", 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7528999999999998e-05 2024-09-13T04:58:15.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7650 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224819+0000", 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75959005800000001, 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01213468, 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7650 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224819+0000", 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224819+0000", 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224819+0000", 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224821+0000", 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0779999999999998e-06 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224827+0000", 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6359999999999999e-06 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224832+0000", 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0889999999999999e-06 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236844+0000", 2024-09-13T04:58:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012012478 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236908+0000", 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3477000000000002e-05 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236954+0000", 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6233000000000003e-05 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7651 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1875968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224837+0000", 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75957197499999995, 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012206452, 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7651 2024-09-13T04:58:15.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224837+0000", 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224837+0000", 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224837+0000", 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224843+0000", 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3859999999999998e-06 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224849+0000", 2024-09-13T04:58:15.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8009999999999998e-06 2024-09-13T04:58:15.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224854+0000", 2024-09-13T04:58:15.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2024-09-13T04:58:15.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.236972+0000", 2024-09-13T04:58:15.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012118063 2024-09-13T04:58:15.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237008+0000", 2024-09-13T04:58:15.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6362999999999997e-05 2024-09-13T04:58:15.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237044+0000", 2024-09-13T04:58:15.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5617999999999997e-05 2024-09-13T04:58:15.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.044 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.044 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7652 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.044 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224858+0000", 2024-09-13T04:58:15.045 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75955100399999997, 2024-09-13T04:58:15.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012284954000000001, 2024-09-13T04:58:15.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7652 2024-09-13T04:58:15.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.046 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224858+0000", 2024-09-13T04:58:15.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224858+0000", 2024-09-13T04:58:15.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224858+0000", 2024-09-13T04:58:15.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224860+0000", 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0999999999999998e-06 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224867+0000", 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5799999999999997e-06 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224872+0000", 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8890000000000001e-06 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237057+0000", 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012185092999999999 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237112+0000", 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5591999999999999e-05 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237143+0000", 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1069999999999999e-05 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7653 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1957888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224876+0000", 2024-09-13T04:58:15.051 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75953294000000005, 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012343638000000001, 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7653 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224876+0000", 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224876+0000", 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224876+0000", 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224878+0000", 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.367e-06 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224884+0000", 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2539999999999998e-06 2024-09-13T04:58:15.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224889+0000", 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4109999999999998e-06 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237156+0000", 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012267472999999999 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237191+0000", 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5116000000000001e-05 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237220+0000", 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8773999999999999e-05 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7654 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224895+0000", 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75951488899999997, 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012424318, 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7654 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224895+0000", 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224895+0000", 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224894+0000", 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224896+0000", 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.979e-06 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224903+0000", 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4999999999999996e-06 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224907+0000", 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6129999999999997e-06 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237238+0000", 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012330288999999999 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237280+0000", 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2540000000000003e-05 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237319+0000", 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8785999999999998e-05 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7655 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 2039808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224912+0000", 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75949784899999995, 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012493746999999999, 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7655 2024-09-13T04:58:15.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224912+0000", 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224912+0000", 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224911+0000", 2024-09-13T04:58:15.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224914+0000", 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9440000000000001e-06 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224920+0000", 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9619999999999998e-06 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224925+0000", 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6079999999999998e-06 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237335+0000", 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012410061999999999 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237375+0000", 2024-09-13T04:58:15.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0664000000000002e-05 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237405+0000", 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9839e-05 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7656 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 2080768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.066 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.224929+0000", 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.75948067100000005, 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012726444, 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7656 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224929+0000", 2024-09-13T04:58:15.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224929+0000", 2024-09-13T04:58:15.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224928+0000", 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224931+0000", 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0899999999999999e-06 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224936+0000", 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7620000000000001e-06 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.224941+0000", 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4900000000000002e-06 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237505+0000", 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012564565 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237585+0000", 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9986000000000003e-05 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.237655+0000", 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9861000000000001e-05 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7785 3.c 3:32021523:::rbd_object_map.112a2c87bb71:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.071 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.337190+0000", 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.64721984200000005, 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011637951000000001, 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7785 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337190+0000", 2024-09-13T04:58:15.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337190+0000", 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337189+0000", 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337192+0000", 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7070000000000001e-06 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337199+0000", 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7000000000000002e-06 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337204+0000", 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2750000000000003e-06 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.343820+0000", 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066156469999999997 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.343907+0000", 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7140999999999998e-05 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.344322+0000", 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041559700000000003 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.345281+0000", 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000958201 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.348735+0000", 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034540600000000001 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.348788+0000", 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3035000000000003e-05 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.348828+0000", 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9910999999999998e-05 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.077 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7786 3.c 3:32021523:::rbd_object_map.112a2c87bb71:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.337211+0000", 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.64719832399999999, 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011732496, 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7786 2024-09-13T04:58:15.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337211+0000", 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337211+0000", 2024-09-13T04:58:15.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337211+0000", 2024-09-13T04:58:15.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337214+0000", 2024-09-13T04:58:15.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.92e-06 2024-09-13T04:58:15.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337220+0000", 2024-09-13T04:58:15.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0800000000000002e-06 2024-09-13T04:58:15.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.337224+0000", 2024-09-13T04:58:15.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0069999999999999e-06 2024-09-13T04:58:15.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.344768+0000", 2024-09-13T04:58:15.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075443539999999996 2024-09-13T04:58:15.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.344813+0000", 2024-09-13T04:58:15.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5124999999999999e-05 2024-09-13T04:58:15.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:58:15.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.344989+0000", 2024-09-13T04:58:15.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000175475 2024-09-13T04:58:15.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:15.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.346014+0000", 2024-09-13T04:58:15.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001024978 2024-09-13T04:58:15.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:15.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.348879+0000", 2024-09-13T04:58:15.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002865051 2024-09-13T04:58:15.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:15.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.348920+0000", 2024-09-13T04:58:15.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1179e-05 2024-09-13T04:58:15.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.348944+0000", 2024-09-13T04:58:15.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3714999999999999e-05 2024-09-13T04:58:15.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.087 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.088 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7824 3.c 3:32021523:::rbd_object_map.112a2c87bb71:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.088 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.364606+0000", 2024-09-13T04:58:15.088 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.61980391999999995, 2024-09-13T04:58:15.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011868605000000001, 2024-09-13T04:58:15.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:15.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7824 2024-09-13T04:58:15.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.089 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364606+0000", 2024-09-13T04:58:15.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364606+0000", 2024-09-13T04:58:15.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364605+0000", 2024-09-13T04:58:15.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364608+0000", 2024-09-13T04:58:15.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5310000000000002e-06 2024-09-13T04:58:15.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364616+0000", 2024-09-13T04:58:15.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4410000000000004e-06 2024-09-13T04:58:15.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364622+0000", 2024-09-13T04:58:15.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3969999999999999e-06 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.370296+0000", 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005673666 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.370391+0000", 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5160000000000004e-05 2024-09-13T04:58:15.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.370681+0000", 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028974799999999999 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.372478+0000", 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001797093 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.376406+0000", 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039279090000000003 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.376440+0000", 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4252999999999999e-05 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.376474+0000", 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3995999999999998e-05 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7825 3.c 3:32021523:::rbd_object_map.112a2c87bb71:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.364636+0000", 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.61977298300000006, 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011940554000000001, 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7825 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364636+0000", 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364636+0000", 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364636+0000", 2024-09-13T04:58:15.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364639+0000", 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.405e-06 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364646+0000", 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.844e-06 2024-09-13T04:58:15.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.364651+0000", 2024-09-13T04:58:15.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1699999999999996e-06 2024-09-13T04:58:15.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.371012+0000", 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063605880000000004 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.371066+0000", 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4292000000000001e-05 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.371296+0000", 2024-09-13T04:58:15.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022941699999999999 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.372483+0000", 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001187182 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.376523+0000", 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040397979999999998 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:15.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.376556+0000", 2024-09-13T04:58:15.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3247000000000002e-05 2024-09-13T04:58:15.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.376577+0000", 2024-09-13T04:58:15.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0993e-05 2024-09-13T04:58:15.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.105 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.105 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.105 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:15.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:15.107 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:15.104+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:15.208 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:15.208+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:15.275 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:15.275 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:15.275 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:15.275 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T04:58:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:58:15.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:15.309 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T04:58:15.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:58:15.310 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:15.308+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:15.401 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone3 from 0xd21cd3d (overlap 0x4eb0459) to 0xfbcdab 2024-09-13T04:58:15.409 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:15.408+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:15.426 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:15.426 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:15.426 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:15.426 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:15.426 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:15.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:15.510 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:15.508+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:15.610 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:15.608+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:15.687 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:15.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:11953 3.10 3:09157e18:::rbd_data.112a2c87bb71.0000000000000060:head [stat,delete] snapc 7=[7] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.581055+0000", 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.105655622, 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045761914000000001, 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 11953 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.581055+0000", 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.581055+0000", 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.581055+0000", 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.581056+0000", 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.319e-06 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.581060+0000", 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0559999999999998e-06 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.581063+0000", 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5399999999999998e-06 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.618262+0000", 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037198888999999999 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.618325+0000", 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2972000000000006e-05 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.618450+0000", 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012496600000000001 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.619706+0000", 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012569129999999999 2024-09-13T04:58:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.626735+0000", 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070288169999999997 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.626779+0000", 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4047e-05 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.626817+0000", 2024-09-13T04:58:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7595000000000002e-05 2024-09-13T04:58:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12285 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.597059+0000", 2024-09-13T04:58:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.089651373000000006, 2024-09-13T04:58:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048634064999999997, 2024-09-13T04:58:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12285 2024-09-13T04:58:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.597059+0000", 2024-09-13T04:58:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.597059+0000", 2024-09-13T04:58:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.597059+0000", 2024-09-13T04:58:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.597061+0000", 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.277e-06 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.597069+0000", 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7719999999999994e-06 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.597074+0000", 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5499999999999996e-06 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.638917+0000", 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041843453000000003 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.639002+0000", 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4745000000000003e-05 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.645693+0000", 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066916019999999996 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12355 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3502080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598625+0000", 2024-09-13T04:58:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.088085425999999994, 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043749821000000001, 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12355 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598625+0000", 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598625+0000", 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598625+0000", 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598627+0000", 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.103e-06 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598634+0000", 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660000000000002e-06 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598641+0000", 2024-09-13T04:58:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2439999999999998e-06 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.642242+0000", 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043600777 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.642305+0000", 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3418999999999997e-05 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.642375+0000", 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0112999999999998e-05 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12356 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3543040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598645+0000", 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.088065604000000006, 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044356007000000003, 2024-09-13T04:58:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12356 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598645+0000", 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598645+0000", 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598645+0000", 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598647+0000", 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9400000000000001e-06 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598654+0000", 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.43e-06 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598659+0000", 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9119999999999997e-06 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.642399+0000", 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043739605000000001 2024-09-13T04:58:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.642932+0000", 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00053334499999999996 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643001+0000", 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9097000000000006e-05 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12357 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3584000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598663+0000", 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.088047685000000001, 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044909038999999998, 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12357 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598663+0000", 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598663+0000", 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598663+0000", 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598665+0000", 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0959999999999999e-06 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598671+0000", 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8590000000000001e-06 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598677+0000", 2024-09-13T04:58:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9959999999999999e-06 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643404+0000", 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044727346000000001 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643495+0000", 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0649999999999994e-05 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643572+0000", 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7423999999999996e-05 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12358 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3624960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598685+0000", 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.088025811999999995, 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045023106, 2024-09-13T04:58:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12358 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598685+0000", 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598685+0000", 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598684+0000", 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598687+0000", 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598693+0000", 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1650000000000003e-06 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598698+0000", 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7559999999999999e-06 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643596+0000", 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044898757999999997 2024-09-13T04:58:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643653+0000", 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6146000000000001e-05 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643708+0000", 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5439000000000002e-05 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12359 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3665920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598702+0000", 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.088009142999999998, 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045101772999999998, 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12359 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598702+0000", 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598702+0000", 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598701+0000", 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598716+0000", 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4528999999999999e-05 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598724+0000", 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3350000000000007e-06 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598731+0000", 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6510000000000004e-06 2024-09-13T04:58:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643723+0000", 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044991997999999998 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643762+0000", 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9048999999999998e-05 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643803+0000", 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1529999999999997e-05 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12360 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3706880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598737+0000", 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087974001999999996, 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045235327999999998, 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12360 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598737+0000", 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598737+0000", 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598736+0000", 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598739+0000", 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4399999999999999e-06 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598746+0000", 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2609999999999996e-06 2024-09-13T04:58:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598772+0000", 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6398e-05 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643824+0000", 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045052166999999997 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643903+0000", 2024-09-13T04:58:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8749000000000001e-05 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643972+0000", 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8779e-05 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12361 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3747840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598778+0000", 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087932801000000005, 2024-09-13T04:58:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045361591, 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12361 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598778+0000", 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598778+0000", 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598777+0000", 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598780+0000", 2024-09-13T04:58:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0790000000000001e-06 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598788+0000", 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8699999999999992e-06 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598793+0000", 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.702e-06 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.643995+0000", 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045201988999999998 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644074+0000", 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8697999999999998e-05 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644139+0000", 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5794000000000001e-05 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12362 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3788800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598797+0000", 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087913395000000005, 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045513773, 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12362 2024-09-13T04:58:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598797+0000", 2024-09-13T04:58:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598797+0000", 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598797+0000", 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598799+0000", 2024-09-13T04:58:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4590000000000001e-06 2024-09-13T04:58:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598807+0000", 2024-09-13T04:58:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7579999999999992e-06 2024-09-13T04:58:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598813+0000", 2024-09-13T04:58:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6910000000000002e-06 2024-09-13T04:58:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644162+0000", 2024-09-13T04:58:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045349234000000002 2024-09-13T04:58:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644238+0000", 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5858000000000005e-05 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644311+0000", 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3116999999999998e-05 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12363 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3829760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598817+0000", 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087893562999999994, 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045692338999999998, 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12363 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598817+0000", 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598817+0000", 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598817+0000", 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598819+0000", 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.745e-06 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598827+0000", 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6799999999999993e-06 2024-09-13T04:58:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598832+0000", 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9389999999999996e-06 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644349+0000", 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045516962000000001 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644429+0000", 2024-09-13T04:58:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0378999999999997e-05 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644509+0000", 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9991e-05 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12364 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3870720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598836+0000", 2024-09-13T04:58:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087874453000000005, 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045844155999999997, 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12364 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598836+0000", 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598836+0000", 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598836+0000", 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598838+0000", 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.962e-06 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598845+0000", 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.764e-06 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598849+0000", 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5709999999999999e-06 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644535+0000", 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045685430999999999 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644608+0000", 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3161e-05 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644680+0000", 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2576000000000004e-05 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12365 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3911680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598853+0000", 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087857735000000006, 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045993538, 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12365 2024-09-13T04:58:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598853+0000", 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598853+0000", 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598853+0000", 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598854+0000", 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9010000000000001e-06 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598861+0000", 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8870000000000003e-06 2024-09-13T04:58:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598865+0000", 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0810000000000004e-06 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644702+0000", 2024-09-13T04:58:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045836659000000002 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644776+0000", 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3512999999999999e-05 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644846+0000", 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0822999999999996e-05 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12366 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3952640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598870+0000", 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087840814000000003, 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046159434999999999, 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12366 2024-09-13T04:58:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598870+0000", 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598870+0000", 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598870+0000", 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598878+0000", 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6889999999999993e-06 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598887+0000", 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3569999999999998e-06 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598891+0000", 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7559999999999999e-06 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644872+0000", 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045980337000000003 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.644951+0000", 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9779999999999998e-05 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.645029+0000", 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7835000000000003e-05 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12367 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 3993600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598896+0000", 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087814608000000002, 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046887312, 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12367 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598896+0000", 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598896+0000", 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598896+0000", 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598898+0000", 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.114e-06 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598908+0000", 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9650000000000003e-06 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598913+0000", 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1440000000000002e-06 2024-09-13T04:58:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.645052+0000", 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046139346999999997 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.645130+0000", 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8157000000000004e-05 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.645783+0000", 2024-09-13T04:58:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00065293299999999996 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12368 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 4034560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598917+0000", 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087793456000000006, 2024-09-13T04:58:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047086099999999999, 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12368 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598917+0000", 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598917+0000", 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598917+0000", 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598919+0000", 2024-09-13T04:58:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:58:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598925+0000", 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1789999999999996e-06 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598930+0000", 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9869999999999999e-06 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.645814+0000", 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046883445000000003 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.645906+0000", 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2268000000000005e-05 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646003+0000", 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7338999999999998e-05 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12369 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 4075520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598935+0000", 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087776034000000003, 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047246541000000003, 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12369 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598935+0000", 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598935+0000", 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598934+0000", 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598936+0000", 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0839999999999999e-06 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598944+0000", 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2910000000000002e-06 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598949+0000", 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.938e-06 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646032+0000", 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047083207000000002 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646113+0000", 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1521999999999996e-05 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646181+0000", 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7884000000000003e-05 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12370 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 4116480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598954+0000", 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087756564999999995, 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047404699000000002, 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12370 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598954+0000", 2024-09-13T04:58:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598954+0000", 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598954+0000", 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598956+0000", 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.238e-06 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598964+0000", 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6070000000000002e-06 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598969+0000", 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.916e-06 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646215+0000", 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047246709999999997 2024-09-13T04:58:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646287+0000", 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2101999999999995e-05 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646359+0000", 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1435999999999998e-05 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12371 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 4157440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.598973+0000", 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.087737387, 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049724376000000001, 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12371 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598973+0000", 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598973+0000", 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.598973+0000", 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.599001+0000", 2024-09-13T04:58:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8503000000000001e-05 2024-09-13T04:58:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.599011+0000", 2024-09-13T04:58:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2909999999999996e-06 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.599016+0000", 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5840000000000002e-06 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646386+0000", 2024-09-13T04:58:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047369745999999997 2024-09-13T04:58:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.646467+0000", 2024-09-13T04:58:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1318999999999998e-05 2024-09-13T04:58:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.648698+0000", 2024-09-13T04:58:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022302849999999998 2024-09-13T04:58:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:12843 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 4182016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.630587+0000", 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.056124038000000001, 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043694326999999998, 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 12843 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.630587+0000", 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.630587+0000", 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.630586+0000", 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.630588+0000", 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1689999999999999e-06 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.630594+0000", 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.630600+0000", 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3580000000000002e-06 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.674121+0000", 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043520956999999999 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.674214+0000", 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3710999999999998e-05 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.674281+0000", 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6660000000000002e-05 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:15.793 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:15.795 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:15.795 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:15.792+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:15.830 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:58:15.851 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-13T04:58:15.279833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107170616,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":13,"num_read":4340,"num_read_kb":4510,"num_write":1201,"num_write_kb":8233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":977,"ondisk_log_size":977,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249244,"kb_used_data":4368,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754725476,"statfs":{"total":773094113280,"available":772838887424,"internally_reserved":0,"allocated":4472832,"data_stored":4277087,"data_compressed":914261,"data_compressed_allocated":1089536,"data_compressed_original":2899968,"omap_allocated":12743,"internal_metadata":250596921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":97783693,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":13,"num_read":4080,"num_read_kb":4293,"num_write":1014,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033537"},"pg_stats":[{"pgid":"3.1f","version":"31'37","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.856359+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:58:11.856359+0000","last_peered":"2024-09-13T04:58:11.856359+0000","last_clean":"2024-09-13T04:58:11.856359+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:58:11.856359+0000","last_undegraded":"2024-09-13T04:58:11.856359+0000","last_fullsized":"2024-09-13T04:58:11.856359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":12582968,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":86,"num_read_kb":343,"num_write":36,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T04:58:10.283200+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:58:10.283200+0000","last_peered":"2024-09-13T04:58:10.283200+0000","last_clean":"2024-09-13T04:58:10.283200+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:58:10.283200+0000","last_undegraded":"2024-09-13T04:58:10.283200+0000","last_fullsized":"2024-09-13T04:58:10.283200+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"31'25","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.105244+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:58:12.105244+0000","last_peered":"2024-09-13T04:58:12.105244+0000","last_clean":"2024-09-13T04:58:12.105244+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:58:12.105244+0000","last_undegraded":"2024-09-13T04:58:12.105244+0000","last_fullsized":"2024-09-13T04:58:12.105244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":11540858,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":17,"num_read_kb":197,"num_write":24,"num_write_kb":460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"32'16","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.455346+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:58:12.455346+0000","last_peered":"2024-09-13T04:58:12.455346+0000","last_clean":"2024-09-13T04:58:12.455346+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:58:12.455346+0000","last_undegraded":"2024-09-13T04:58:12.455346+0000","last_fullsized":"2024-09-13T04:58:12.455346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":16,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"32'17","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.537445+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:58:12.537445+0000","last_peered":"2024-09-13T04:58:12.537445+0000","last_clean":"2024-09-13T04:58:12.537445+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:58:12.537445+0000","last_undegraded":"2024-09-13T04:58:12.537445+0000","last_fullsized":"2024-09-13T04:58:12.537445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":32,"num_write":18,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"32'20","reported_seq":179,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.606739+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:58:12.606739+0000","last_peered":"2024-09-13T04:58:12.606739+0000","last_clean":"2024-09-13T04:58:12.606739+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:58:12.606739+0000","last_undegraded":"2024-09-13T04:58:12.606739+0000","last_fullsized":"2024-09-13T04:58:12.606739+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":6830680,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":152,"num_read_kb":219,"num_write":20,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"32'25","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.592291+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:58:12.592291+0000","last_peered":"2024-09-13T04:58:12.592291+0000","last_clean":"2024-09-13T04:58:12.592291+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:58:12.592291+0000","last_undegraded":"2024-09-13T04:58:12.592291+0000","last_fullsized":"2024-09-13T04:58:12.592291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":3380112,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":39,"num_write":24,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"31'31","reported_seq":85,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.851591+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:58:11.851591+0000","last_peered":"2024-09-13T04:58:11.851591+0000","last_clean":"2024-09-13T04:58:11.851591+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:58:11.851591+0000","last_undegraded":"2024-09-13T04:58:11.851591+0000","last_fullsized":"2024-09-13T04:58:11.851591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":14680107,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":70,"num_read_kb":67,"num_write":30,"num_write_kb":177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"31'3","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.826876+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:58:11.826876+0000","last_peered":"2024-09-13T04:58:11.826876+0000","last_clean":"2024-09-13T04:58:11.826876+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:58:11.826876+0000","last_undegraded":"2024-09-13T04:58:11.826876+0000","last_fullsized":"2024-09-13T04:58:11.826876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":3376261,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"31'206","reported_seq":230,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.788683+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:58:11.788683+0000","last_peered":"2024-09-13T04:58:11.788683+0000","last_clean":"2024-09-13T04:58:11.788683+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:58:11.788683+0000","last_undegraded":"2024-09-13T04:58:11.788683+0000","last_fullsized":"2024-09-13T04:58:11.788683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":1048606,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":949,"num_read_kb":749,"num_write":325,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"32'4","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.307754+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:58:12.307754+0000","last_peered":"2024-09-13T04:58:12.307754+0000","last_clean":"2024-09-13T04:58:12.307754+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:58:12.307754+0000","last_undegraded":"2024-09-13T04:58:12.307754+0000","last_fullsized":"2024-09-13T04:58:12.307754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"32'4","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.477885+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:58:12.477885+0000","last_peered":"2024-09-13T04:58:12.477885+0000","last_clean":"2024-09-13T04:58:12.477885+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:58:12.477885+0000","last_undegraded":"2024-09-13T04:58:12.477885+0000","last_fullsized":"2024-09-13T04:58:12.477885+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":5,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252694+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:58:11.252694+0000","last_peered":"2024-09-13T04:58:11.252694+0000","last_clean":"2024-09-13T04:58:11.252694+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:58:11.252694+0000","last_undegraded":"2024-09-13T04:58:11.252694+0000","last_fullsized":"2024-09-13T04:58:11.252694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'2","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.317984+0000","last_change":"2024-09-13T04:58:05.912872+0000","last_active":"2024-09-13T04:58:12.317984+0000","last_peered":"2024-09-13T04:58:12.317984+0000","last_clean":"2024-09-13T04:58:12.317984+0000","last_became_active":"2024-09-13T04:58:03.926978+0000","last_became_peered":"2024-09-13T04:58:03.926978+0000","last_unstale":"2024-09-13T04:58:12.317984+0000","last_undegraded":"2024-09-13T04:58:12.317984+0000","last_fullsized":"2024-09-13T04:58:12.317984+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:30:43.308903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.250466+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:58:11.250466+0000","last_peered":"2024-09-13T04:58:11.250466+0000","last_clean":"2024-09-13T04:58:11.250466+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:58:11.250466+0000","last_undegraded":"2024-09-13T04:58:11.250466+0000","last_fullsized":"2024-09-13T04:58:11.250466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"32'18","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.317262+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:58:12.317262+0000","last_peered":"2024-09-13T04:58:12.317262+0000","last_clean":"2024-09-13T04:58:12.317262+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:58:12.317262+0000","last_undegraded":"2024-09-13T04:58:12.317262+0000","last_fullsized":"2024-09-13T04:58:12.317262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":18,"num_write_kb":178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252550+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:58:11.252550+0000","last_peered":"2024-09-13T04:58:11.252550+0000","last_clean":"2024-09-13T04:58:11.252550+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:58:11.252550+0000","last_undegraded":"2024-09-13T04:58:11.252550+0000","last_fullsized":"2024-09-13T04:58:11.252550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"32'5","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.396756+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:58:12.396756+0000","last_peered":"2024-09-13T04:58:12.396756+0000","last_clean":"2024-09-13T04:58:12.396756+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:58:12.396756+0000","last_undegraded":"2024-09-13T04:58:12.396756+0000","last_fullsized":"2024-09-13T04:58:12.396756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T04:58:10.283182+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:58:10.283182+0000","last_peered":"2024-09-13T04:58:10.283182+0000","last_clean":"2024-09-13T04:58:10.283182+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:58:10.283182+0000","last_undegraded":"2024-09-13T04:58:10.283182+0000","last_fullsized":"2024-09-13T04:58:10.283182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"30'14","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.249558+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:58:11.249558+0000","last_peered":"2024-09-13T04:58:11.249558+0000","last_clean":"2024-09-13T04:58:11.249558+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:58:11.249558+0000","last_undegraded":"2024-09-13T04:58:11.249558+0000","last_fullsized":"2024-09-13T04:58:11.249558+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":14,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"32'26","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.598971+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:58:12.598971+0000","last_peered":"2024-09-13T04:58:12.598971+0000","last_clean":"2024-09-13T04:58:12.598971+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:58:12.598971+0000","last_undegraded":"2024-09-13T04:58:12.598971+0000","last_fullsized":"2024-09-13T04:58:12.598971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":6232992,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":4,"num_read_kb":0,"num_write":25,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252384+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:58:11.252384+0000","last_peered":"2024-09-13T04:58:11.252384+0000","last_clean":"2024-09-13T04:58:11.252384+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:58:11.252384+0000","last_undegraded":"2024-09-13T04:58:11.252384+0000","last_fullsized":"2024-09-13T04:58:11.252384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"31'16","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.373080+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:58:11.373080+0000","last_peered":"2024-09-13T04:58:11.373080+0000","last_clean":"2024-09-13T04:58:11.373080+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:58:11.373080+0000","last_undegraded":"2024-09-13T04:58:11.373080+0000","last_fullsized":"2024-09-13T04:58:11.373080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":16,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.310012+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:58:11.310012+0000","last_peered":"2024-09-13T04:58:11.310012+0000","last_clean":"2024-09-13T04:58:11.310012+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:58:11.310012+0000","last_undegraded":"2024-09-13T04:58:11.310012+0000","last_fullsized":"2024-09-13T04:58:11.310012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"31'23","reported_seq":73,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.843783+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:58:11.843783+0000","last_peered":"2024-09-13T04:58:11.843783+0000","last_clean":"2024-09-13T04:58:11.843783+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:58:11.843783+0000","last_undegraded":"2024-09-13T04:58:11.843783+0000","last_fullsized":"2024-09-13T04:58:11.843783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":7636017,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":46,"num_read_kb":264,"num_write":22,"num_write_kb":374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252077+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:58:11.252077+0000","last_peered":"2024-09-13T04:58:11.252077+0000","last_clean":"2024-09-13T04:58:11.252077+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:58:11.252077+0000","last_undegraded":"2024-09-13T04:58:11.252077+0000","last_fullsized":"2024-09-13T04:58:11.252077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"28'1","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.250473+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:58:11.250473+0000","last_peered":"2024-09-13T04:58:11.250473+0000","last_clean":"2024-09-13T04:58:11.250473+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:58:11.250473+0000","last_undegraded":"2024-09-13T04:58:11.250473+0000","last_fullsized":"2024-09-13T04:58:11.250473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252377+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:58:11.252377+0000","last_peered":"2024-09-13T04:58:11.252377+0000","last_clean":"2024-09-13T04:58:11.252377+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:58:11.252377+0000","last_undegraded":"2024-09-13T04:58:11.252377+0000","last_fullsized":"2024-09-13T04:58:11.252377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.248127+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:58:11.248127+0000","last_peered":"2024-09-13T04:58:11.248127+0000","last_clean":"2024-09-13T04:58:11.248127+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:58:11.248127+0000","last_undegraded":"2024-09-13T04:58:11.248127+0000","last_fullsized":"2024-09-13T04:58:11.248127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"31'13","reported_seq":142,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.616682+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:58:11.616682+0000","last_peered":"2024-09-13T04:58:11.616682+0000","last_clean":"2024-09-13T04:58:11.616682+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:58:11.616682+0000","last_undegraded":"2024-09-13T04:58:11.616682+0000","last_fullsized":"2024-09-13T04:58:11.616682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":330,"num_read_kb":284,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"32'108","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.627004+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:58:12.627004+0000","last_peered":"2024-09-13T04:58:12.627004+0000","last_clean":"2024-09-13T04:58:12.627004+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:58:12.627004+0000","last_undegraded":"2024-09-13T04:58:12.627004+0000","last_fullsized":"2024-09-13T04:58:12.627004+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":4162346,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":468,"num_read_kb":371,"num_write":172,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"32'26","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.477909+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:58:12.477909+0000","last_peered":"2024-09-13T04:58:12.477909+0000","last_clean":"2024-09-13T04:58:12.477909+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:58:12.477909+0000","last_undegraded":"2024-09-13T04:58:12.477909+0000","last_fullsized":"2024-09-13T04:58:12.477909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":27,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.310045+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:58:11.310045+0000","last_peered":"2024-09-13T04:58:11.310045+0000","last_clean":"2024-09-13T04:58:11.310045+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:58:11.310045+0000","last_undegraded":"2024-09-13T04:58:11.310045+0000","last_fullsized":"2024-09-13T04:58:11.310045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"28'11","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.306081+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:58:11.306081+0000","last_peered":"2024-09-13T04:58:11.306081+0000","last_clean":"2024-09-13T04:58:11.306081+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:58:11.306081+0000","last_undegraded":"2024-09-13T04:58:11.306081+0000","last_fullsized":"2024-09-13T04:58:11.306081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":43,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":31,"num_write":15,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"31'103","reported_seq":636,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.789549+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:58:11.789549+0000","last_peered":"2024-09-13T04:58:11.789549+0000","last_clean":"2024-09-13T04:58:11.789549+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:58:11.789549+0000","last_undegraded":"2024-09-13T04:58:11.789549+0000","last_fullsized":"2024-09-13T04:58:11.789549+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":2413040,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":1420,"num_read_kb":1256,"num_write":106,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"31'15","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.355115+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:58:11.355115+0000","last_peered":"2024-09-13T04:58:11.355115+0000","last_clean":"2024-09-13T04:58:11.355115+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:58:11.355115+0000","last_undegraded":"2024-09-13T04:58:11.355115+0000","last_fullsized":"2024-09-13T04:58:11.355115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":4492964,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":14,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"32'8","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.534988+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:58:12.534988+0000","last_peered":"2024-09-13T04:58:12.534988+0000","last_clean":"2024-09-13T04:58:12.534988+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:58:12.534988+0000","last_undegraded":"2024-09-13T04:58:12.534988+0000","last_fullsized":"2024-09-13T04:58:12.534988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":3145728,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"32'10","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.537525+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:58:12.537525+0000","last_peered":"2024-09-13T04:58:12.537525+0000","last_clean":"2024-09-13T04:58:12.537525+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:58:12.537525+0000","last_undegraded":"2024-09-13T04:58:12.537525+0000","last_fullsized":"2024-09-13T04:58:12.537525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":3145728,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":28,"num_write":10,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"32'30","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.495086+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:58:12.495086+0000","last_peered":"2024-09-13T04:58:12.495086+0000","last_clean":"2024-09-13T04:58:12.495086+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:58:12.495086+0000","last_undegraded":"2024-09-13T04:58:12.495086+0000","last_fullsized":"2024-09-13T04:58:12.495086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":9396671,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":42,"num_read_kb":52,"num_write":30,"num_write_kb":439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"32'7","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.515926+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:58:12.515926+0000","last_peered":"2024-09-13T04:58:12.515926+0000","last_clean":"2024-09-13T04:58:12.515926+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:58:12.515926+0000","last_undegraded":"2024-09-13T04:58:12.515926+0000","last_fullsized":"2024-09-13T04:58:12.515926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":36,"num_write":8,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"32'33","reported_seq":234,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.641714+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:58:12.641714+0000","last_peered":"2024-09-13T04:58:12.641714+0000","last_clean":"2024-09-13T04:58:12.641714+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:58:12.641714+0000","last_undegraded":"2024-09-13T04:58:12.641714+0000","last_fullsized":"2024-09-13T04:58:12.641714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":3077892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":483,"num_read_kb":422,"num_write":33,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":106580229,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":13,"num_read":4224,"num_read_kb":4410,"num_write":1040,"num_write_kb":5815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1830912,"data_stored":2249236,"data_compressed":1015913,"data_compressed_allocated":1146880,"data_compressed_original":1925120,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":127535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":374306,"data_compressed":124778,"data_compressed_allocated":135168,"data_compressed_original":196608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":413696,"data_stored":218155,"data_compressed":40766,"data_compressed_allocated":49152,"data_compressed_original":77824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":737280,"data_stored":622226,"data_compressed":200816,"data_compressed_allocated":225280,"data_compressed_original":409600,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35768,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94336072,"statfs":{"total":96636764160,"available":96600137728,"internally_reserved":0,"allocated":569344,"data_stored":394413,"data_compressed":149112,"data_compressed_allocated":155648,"data_compressed_original":225280,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":696320,"data_stored":1134811,"data_compressed":163978,"data_compressed_allocated":221184,"data_compressed_original":933888,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":528384,"data_stored":402170,"data_compressed":103360,"data_compressed_allocated":118784,"data_compressed_original":237568,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":626688,"data_stored":1003471,"data_compressed":131451,"data_compressed_allocated":184320,"data_compressed_original":819200,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":196861,"data_compressed":100496,"data_compressed_allocated":110592,"data_compressed_original":167936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":452694,"data_compressed":179459,"data_compressed_allocated":208896,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":310099,"data_compressed":179010,"data_compressed_allocated":188416,"data_compressed_original":270336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":507366,"data_compressed":190178,"data_compressed_allocated":221184,"data_compressed_original":430080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":160942,"data_compressed":84153,"data_compressed_allocated":98304,"data_compressed_original":143360,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":380213,"data_compressed":157839,"data_compressed_allocated":184320,"data_compressed_original":323584,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":241002,"data_compressed":124778,"data_compressed_allocated":135168,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":59,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:58:15.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:58:15.937 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:15.936+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:15.953 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:15.953 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:15.968 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:15.984 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-13T04:58:15.991 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:15.995 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:15.996+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:16.006 INFO:teuthology.orchestra.run.smithi001.stdout:753 write 0xdcabb63 thru 0xdcb790b (0xbda9 bytes) 2024-09-13T04:58:16.012 INFO:teuthology.orchestra.run.smithi001.stdout:755 punch from 0x518ec5a to 0x518f843, (0xbe9 bytes) 2024-09-13T04:58:16.012 INFO:teuthology.orchestra.run.smithi001.stdout:756 write 0xb67de9d thru 0xb68d9ea (0xfb4e bytes) 2024-09-13T04:58:16.096 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:16.092+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:16.140 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:16.140 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:16.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:16.197 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:16.196+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:16.245 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:58:16.259 INFO:teuthology.orchestra.run.smithi001.stdout:757 punch from 0xa8fd6dd to 0xa90cd7e, (0xf6a1 bytes) 2024-09-13T04:58:16.259 INFO:teuthology.orchestra.run.smithi001.stdout:758 punch from 0x12ce73a to 0x12d58c3, (0x7189 bytes) 2024-09-13T04:58:16.262 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-13T04:58:15.279833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107170616,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":13,"num_read":4340,"num_read_kb":4510,"num_write":1201,"num_write_kb":8233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":977,"ondisk_log_size":977,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249244,"kb_used_data":4368,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754725476,"statfs":{"total":773094113280,"available":772838887424,"internally_reserved":0,"allocated":4472832,"data_stored":4277087,"data_compressed":914261,"data_compressed_allocated":1089536,"data_compressed_original":2899968,"omap_allocated":12743,"internal_metadata":250596921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":97783693,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":13,"num_read":4080,"num_read_kb":4293,"num_write":1014,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033537"},"pg_stats":[{"pgid":"3.1f","version":"31'37","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.856359+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:58:11.856359+0000","last_peered":"2024-09-13T04:58:11.856359+0000","last_clean":"2024-09-13T04:58:11.856359+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:58:11.856359+0000","last_undegraded":"2024-09-13T04:58:11.856359+0000","last_fullsized":"2024-09-13T04:58:11.856359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":12582968,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":86,"num_read_kb":343,"num_write":36,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T04:58:10.283200+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:58:10.283200+0000","last_peered":"2024-09-13T04:58:10.283200+0000","last_clean":"2024-09-13T04:58:10.283200+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:58:10.283200+0000","last_undegraded":"2024-09-13T04:58:10.283200+0000","last_fullsized":"2024-09-13T04:58:10.283200+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"31'25","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.105244+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:58:12.105244+0000","last_peered":"2024-09-13T04:58:12.105244+0000","last_clean":"2024-09-13T04:58:12.105244+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:58:12.105244+0000","last_undegraded":"2024-09-13T04:58:12.105244+0000","last_fullsized":"2024-09-13T04:58:12.105244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":11540858,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":17,"num_read_kb":197,"num_write":24,"num_write_kb":460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"32'16","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.455346+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:58:12.455346+0000","last_peered":"2024-09-13T04:58:12.455346+0000","last_clean":"2024-09-13T04:58:12.455346+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:58:12.455346+0000","last_undegraded":"2024-09-13T04:58:12.455346+0000","last_fullsized":"2024-09-13T04:58:12.455346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":16,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"32'17","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.537445+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:58:12.537445+0000","last_peered":"2024-09-13T04:58:12.537445+0000","last_clean":"2024-09-13T04:58:12.537445+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:58:12.537445+0000","last_undegraded":"2024-09-13T04:58:12.537445+0000","last_fullsized":"2024-09-13T04:58:12.537445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":32,"num_write":18,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"32'20","reported_seq":179,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.606739+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:58:12.606739+0000","last_peered":"2024-09-13T04:58:12.606739+0000","last_clean":"2024-09-13T04:58:12.606739+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:58:12.606739+0000","last_undegraded":"2024-09-13T04:58:12.606739+0000","last_fullsized":"2024-09-13T04:58:12.606739+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":6830680,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":152,"num_read_kb":219,"num_write":20,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"32'25","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.592291+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:58:12.592291+0000","last_peered":"2024-09-13T04:58:12.592291+0000","last_clean":"2024-09-13T04:58:12.592291+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:58:12.592291+0000","last_undegraded":"2024-09-13T04:58:12.592291+0000","last_fullsized":"2024-09-13T04:58:12.592291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":3380112,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":39,"num_write":24,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"31'31","reported_seq":85,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.851591+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:58:11.851591+0000","last_peered":"2024-09-13T04:58:11.851591+0000","last_clean":"2024-09-13T04:58:11.851591+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:58:11.851591+0000","last_undegraded":"2024-09-13T04:58:11.851591+0000","last_fullsized":"2024-09-13T04:58:11.851591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":14680107,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":70,"num_read_kb":67,"num_write":30,"num_write_kb":177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"31'3","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.826876+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:58:11.826876+0000","last_peered":"2024-09-13T04:58:11.826876+0000","last_clean":"2024-09-13T04:58:11.826876+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:58:11.826876+0000","last_undegraded":"2024-09-13T04:58:11.826876+0000","last_fullsized":"2024-09-13T04:58:11.826876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":3376261,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"31'206","reported_seq":230,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.788683+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:58:11.788683+0000","last_peered":"2024-09-13T04:58:11.788683+0000","last_clean":"2024-09-13T04:58:11.788683+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:58:11.788683+0000","last_undegraded":"2024-09-13T04:58:11.788683+0000","last_fullsized":"2024-09-13T04:58:11.788683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":1048606,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":949,"num_read_kb":749,"num_write":325,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"32'4","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.307754+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:58:12.307754+0000","last_peered":"2024-09-13T04:58:12.307754+0000","last_clean":"2024-09-13T04:58:12.307754+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:58:12.307754+0000","last_undegraded":"2024-09-13T04:58:12.307754+0000","last_fullsized":"2024-09-13T04:58:12.307754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"32'4","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.477885+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:58:12.477885+0000","last_peered":"2024-09-13T04:58:12.477885+0000","last_clean":"2024-09-13T04:58:12.477885+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:58:12.477885+0000","last_undegraded":"2024-09-13T04:58:12.477885+0000","last_fullsized":"2024-09-13T04:58:12.477885+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":5,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252694+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:58:11.252694+0000","last_peered":"2024-09-13T04:58:11.252694+0000","last_clean":"2024-09-13T04:58:11.252694+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:58:11.252694+0000","last_undegraded":"2024-09-13T04:58:11.252694+0000","last_fullsized":"2024-09-13T04:58:11.252694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'2","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.317984+0000","last_change":"2024-09-13T04:58:05.912872+0000","last_active":"2024-09-13T04:58:12.317984+0000","last_peered":"2024-09-13T04:58:12.317984+0000","last_clean":"2024-09-13T04:58:12.317984+0000","last_became_active":"2024-09-13T04:58:03.926978+0000","last_became_peered":"2024-09-13T04:58:03.926978+0000","last_unstale":"2024-09-13T04:58:12.317984+0000","last_undegraded":"2024-09-13T04:58:12.317984+0000","last_fullsized":"2024-09-13T04:58:12.317984+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:30:43.308903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.250466+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:58:11.250466+0000","last_peered":"2024-09-13T04:58:11.250466+0000","last_clean":"2024-09-13T04:58:11.250466+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:58:11.250466+0000","last_undegraded":"2024-09-13T04:58:11.250466+0000","last_fullsized":"2024-09-13T04:58:11.250466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"32'18","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.317262+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:58:12.317262+0000","last_peered":"2024-09-13T04:58:12.317262+0000","last_clean":"2024-09-13T04:58:12.317262+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:58:12.317262+0000","last_undegraded":"2024-09-13T04:58:12.317262+0000","last_fullsized":"2024-09-13T04:58:12.317262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":18,"num_write_kb":178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252550+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:58:11.252550+0000","last_peered":"2024-09-13T04:58:11.252550+0000","last_clean":"2024-09-13T04:58:11.252550+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:58:11.252550+0000","last_undegraded":"2024-09-13T04:58:11.252550+0000","last_fullsized":"2024-09-13T04:58:11.252550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"32'5","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.396756+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:58:12.396756+0000","last_peered":"2024-09-13T04:58:12.396756+0000","last_clean":"2024-09-13T04:58:12.396756+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:58:12.396756+0000","last_undegraded":"2024-09-13T04:58:12.396756+0000","last_fullsized":"2024-09-13T04:58:12.396756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T04:58:10.283182+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:58:10.283182+0000","last_peered":"2024-09-13T04:58:10.283182+0000","last_clean":"2024-09-13T04:58:10.283182+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:58:10.283182+0000","last_undegraded":"2024-09-13T04:58:10.283182+0000","last_fullsized":"2024-09-13T04:58:10.283182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"30'14","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.249558+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:58:11.249558+0000","last_peered":"2024-09-13T04:58:11.249558+0000","last_clean":"2024-09-13T04:58:11.249558+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:58:11.249558+0000","last_undegraded":"2024-09-13T04:58:11.249558+0000","last_fullsized":"2024-09-13T04:58:11.249558+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":14,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"32'26","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.598971+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:58:12.598971+0000","last_peered":"2024-09-13T04:58:12.598971+0000","last_clean":"2024-09-13T04:58:12.598971+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:58:12.598971+0000","last_undegraded":"2024-09-13T04:58:12.598971+0000","last_fullsized":"2024-09-13T04:58:12.598971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":6232992,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":4,"num_read_kb":0,"num_write":25,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252384+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:58:11.252384+0000","last_peered":"2024-09-13T04:58:11.252384+0000","last_clean":"2024-09-13T04:58:11.252384+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:58:11.252384+0000","last_undegraded":"2024-09-13T04:58:11.252384+0000","last_fullsized":"2024-09-13T04:58:11.252384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"31'16","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.373080+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:58:11.373080+0000","last_peered":"2024-09-13T04:58:11.373080+0000","last_clean":"2024-09-13T04:58:11.373080+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:58:11.373080+0000","last_undegraded":"2024-09-13T04:58:11.373080+0000","last_fullsized":"2024-09-13T04:58:11.373080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":16,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.310012+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:58:11.310012+0000","last_peered":"2024-09-13T04:58:11.310012+0000","last_clean":"2024-09-13T04:58:11.310012+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:58:11.310012+0000","last_undegraded":"2024-09-13T04:58:11.310012+0000","last_fullsized":"2024-09-13T04:58:11.310012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"31'23","reported_seq":73,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.843783+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:58:11.843783+0000","last_peered":"2024-09-13T04:58:11.843783+0000","last_clean":"2024-09-13T04:58:11.843783+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:58:11.843783+0000","last_undegraded":"2024-09-13T04:58:11.843783+0000","last_fullsized":"2024-09-13T04:58:11.843783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":7636017,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":46,"num_read_kb":264,"num_write":22,"num_write_kb":374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252077+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:58:11.252077+0000","last_peered":"2024-09-13T04:58:11.252077+0000","last_clean":"2024-09-13T04:58:11.252077+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:58:11.252077+0000","last_undegraded":"2024-09-13T04:58:11.252077+0000","last_fullsized":"2024-09-13T04:58:11.252077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"28'1","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.250473+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:58:11.250473+0000","last_peered":"2024-09-13T04:58:11.250473+0000","last_clean":"2024-09-13T04:58:11.250473+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:58:11.250473+0000","last_undegraded":"2024-09-13T04:58:11.250473+0000","last_fullsized":"2024-09-13T04:58:11.250473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252377+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:58:11.252377+0000","last_peered":"2024-09-13T04:58:11.252377+0000","last_clean":"2024-09-13T04:58:11.252377+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:58:11.252377+0000","last_undegraded":"2024-09-13T04:58:11.252377+0000","last_fullsized":"2024-09-13T04:58:11.252377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.248127+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:58:11.248127+0000","last_peered":"2024-09-13T04:58:11.248127+0000","last_clean":"2024-09-13T04:58:11.248127+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:58:11.248127+0000","last_undegraded":"2024-09-13T04:58:11.248127+0000","last_fullsized":"2024-09-13T04:58:11.248127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"31'13","reported_seq":142,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.616682+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:58:11.616682+0000","last_peered":"2024-09-13T04:58:11.616682+0000","last_clean":"2024-09-13T04:58:11.616682+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:58:11.616682+0000","last_undegraded":"2024-09-13T04:58:11.616682+0000","last_fullsized":"2024-09-13T04:58:11.616682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":330,"num_read_kb":284,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"32'108","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.627004+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:58:12.627004+0000","last_peered":"2024-09-13T04:58:12.627004+0000","last_clean":"2024-09-13T04:58:12.627004+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:58:12.627004+0000","last_undegraded":"2024-09-13T04:58:12.627004+0000","last_fullsized":"2024-09-13T04:58:12.627004+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":4162346,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":468,"num_read_kb":371,"num_write":172,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"32'26","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.477909+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:58:12.477909+0000","last_peered":"2024-09-13T04:58:12.477909+0000","last_clean":"2024-09-13T04:58:12.477909+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:58:12.477909+0000","last_undegraded":"2024-09-13T04:58:12.477909+0000","last_fullsized":"2024-09-13T04:58:12.477909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":27,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.310045+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:58:11.310045+0000","last_peered":"2024-09-13T04:58:11.310045+0000","last_clean":"2024-09-13T04:58:11.310045+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:58:11.310045+0000","last_undegraded":"2024-09-13T04:58:11.310045+0000","last_fullsized":"2024-09-13T04:58:11.310045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"28'11","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.306081+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:58:11.306081+0000","last_peered":"2024-09-13T04:58:11.306081+0000","last_clean":"2024-09-13T04:58:11.306081+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:58:11.306081+0000","last_undegraded":"2024-09-13T04:58:11.306081+0000","last_fullsized":"2024-09-13T04:58:11.306081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":43,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":31,"num_write":15,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"31'103","reported_seq":636,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.789549+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:58:11.789549+0000","last_peered":"2024-09-13T04:58:11.789549+0000","last_clean":"2024-09-13T04:58:11.789549+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:58:11.789549+0000","last_undegraded":"2024-09-13T04:58:11.789549+0000","last_fullsized":"2024-09-13T04:58:11.789549+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":2413040,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":1420,"num_read_kb":1256,"num_write":106,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"31'15","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.355115+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:58:11.355115+0000","last_peered":"2024-09-13T04:58:11.355115+0000","last_clean":"2024-09-13T04:58:11.355115+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:58:11.355115+0000","last_undegraded":"2024-09-13T04:58:11.355115+0000","last_fullsized":"2024-09-13T04:58:11.355115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":4492964,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":14,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"32'8","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.534988+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:58:12.534988+0000","last_peered":"2024-09-13T04:58:12.534988+0000","last_clean":"2024-09-13T04:58:12.534988+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:58:12.534988+0000","last_undegraded":"2024-09-13T04:58:12.534988+0000","last_fullsized":"2024-09-13T04:58:12.534988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":3145728,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"32'10","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.537525+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:58:12.537525+0000","last_peered":"2024-09-13T04:58:12.537525+0000","last_clean":"2024-09-13T04:58:12.537525+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:58:12.537525+0000","last_undegraded":"2024-09-13T04:58:12.537525+0000","last_fullsized":"2024-09-13T04:58:12.537525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":3145728,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":28,"num_write":10,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"32'30","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.495086+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:58:12.495086+0000","last_peered":"2024-09-13T04:58:12.495086+0000","last_clean":"2024-09-13T04:58:12.495086+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:58:12.495086+0000","last_undegraded":"2024-09-13T04:58:12.495086+0000","last_fullsized":"2024-09-13T04:58:12.495086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":9396671,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":42,"num_read_kb":52,"num_write":30,"num_write_kb":439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"32'7","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.515926+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:58:12.515926+0000","last_peered":"2024-09-13T04:58:12.515926+0000","last_clean":"2024-09-13T04:58:12.515926+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:58:12.515926+0000","last_undegraded":"2024-09-13T04:58:12.515926+0000","last_fullsized":"2024-09-13T04:58:12.515926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":36,"num_write":8,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"32'33","reported_seq":234,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.641714+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:58:12.641714+0000","last_peered":"2024-09-13T04:58:12.641714+0000","last_clean":"2024-09-13T04:58:12.641714+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:58:12.641714+0000","last_undegraded":"2024-09-13T04:58:12.641714+0000","last_fullsized":"2024-09-13T04:58:12.641714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":3077892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":483,"num_read_kb":422,"num_write":33,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":106580229,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":13,"num_read":4224,"num_read_kb":4410,"num_write":1040,"num_write_kb":5815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1830912,"data_stored":2249236,"data_compressed":1015913,"data_compressed_allocated":1146880,"data_compressed_original":1925120,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":127535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":374306,"data_compressed":124778,"data_compressed_allocated":135168,"data_compressed_original":196608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":413696,"data_stored":218155,"data_compressed":40766,"data_compressed_allocated":49152,"data_compressed_original":77824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":737280,"data_stored":622226,"data_compressed":200816,"data_compressed_allocated":225280,"data_compressed_original":409600,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35768,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94336072,"statfs":{"total":96636764160,"available":96600137728,"internally_reserved":0,"allocated":569344,"data_stored":394413,"data_compressed":149112,"data_compressed_allocated":155648,"data_compressed_original":225280,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":696320,"data_stored":1134811,"data_compressed":163978,"data_compressed_allocated":221184,"data_compressed_original":933888,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":528384,"data_stored":402170,"data_compressed":103360,"data_compressed_allocated":118784,"data_compressed_original":237568,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":626688,"data_stored":1003471,"data_compressed":131451,"data_compressed_allocated":184320,"data_compressed_original":819200,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":196861,"data_compressed":100496,"data_compressed_allocated":110592,"data_compressed_original":167936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":452694,"data_compressed":179459,"data_compressed_allocated":208896,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":310099,"data_compressed":179010,"data_compressed_allocated":188416,"data_compressed_original":270336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":507366,"data_compressed":190178,"data_compressed_allocated":221184,"data_compressed_original":430080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":160942,"data_compressed":84153,"data_compressed_allocated":98304,"data_compressed_original":143360,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":380213,"data_compressed":157839,"data_compressed_allocated":184320,"data_compressed_original":323584,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":241002,"data_compressed":124778,"data_compressed_allocated":135168,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":59,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:58:16.264 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T04:58:16.264 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:58:16.265 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T04:58:16.265 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T04:58:16.268 INFO:teuthology.orchestra.run.smithi001.stdout:761 write 0xe117231 thru 0xe126e93 (0xfc63 bytes) 2024-09-13T04:58:16.274 INFO:teuthology.orchestra.run.smithi001.stdout:762 read 0xd10888f thru 0xd11568d (0xcdff bytes) 2024-09-13T04:58:16.274 INFO:teuthology.orchestra.run.smithi001.stdout:763 read 0xa7d67a7 thru 0xa7e3812 (0xd06c bytes) 2024-09-13T04:58:16.274 INFO:teuthology.orchestra.run.smithi001.stdout:764 punch from 0x118f890 to 0x119a8d0, (0xb040 bytes) 2024-09-13T04:58:16.274 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:16.274 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:16.274 INFO:teuthology.orchestra.run.smithi001.stdout:767 punch from 0x26e2c08 to 0x26e801a, (0x5412 bytes) 2024-09-13T04:58:16.275 INFO:teuthology.orchestra.run.smithi001.stdout:768 write 0xdb9b7de thru 0xdba8b52 (0xd375 bytes) 2024-09-13T04:58:16.280 INFO:teuthology.orchestra.run.smithi001.stdout:769 write 0xaf109b8 thru 0xaf198ff (0x8f48 bytes) 2024-09-13T04:58:16.299 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:16.296+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:16.316 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:16.334 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:761 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.825547+0000", 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.5066523400000005, 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039280260000000003, 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 761 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825547+0000", 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825547+0000", 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825545+0000", 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825558+0000", 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825588+0000", 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.942e-05 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825602+0000", 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4667e-05 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825731+0000", 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012861599999999999 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825883+0000", 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015214199999999999 2024-09-13T04:58:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.826532+0000", 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00064878799999999997 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.828352+0000", 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018203939999999999 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.829391+0000", 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001039166 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.829429+0000", 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7750000000000003e-05 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.829475+0000", 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6109999999999997e-05 2024-09-13T04:58:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:762 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.825659+0000", 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.5065407510000002, 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004818073, 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825659+0000", 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825659+0000", 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825658+0000", 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825679+0000", 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1508000000000001e-05 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825687+0000", 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8839999999999994e-06 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.825692+0000", 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2820000000000004e-06 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.827260+0000", 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001567496 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.827382+0000", 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122392 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.827921+0000", 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00053902599999999996 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.828803+0000", 2024-09-13T04:58:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00088165400000000005 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830328+0000", 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015251399999999999 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830413+0000", 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5390999999999996e-05 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.830477+0000", 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3535999999999996e-05 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:768 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.905819+0000", 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.4263806260000003, 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003311984, 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 768 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.905819+0000", 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.905819+0000", 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.905815+0000", 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999967 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.905839+0000", 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3269999999999999e-05 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.905877+0000", 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8396000000000003e-05 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.905896+0000", 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.853e-05 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.906009+0000", 2024-09-13T04:58:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011283200000000001 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.906188+0000", 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017943599999999999 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.906671+0000", 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00048349899999999997 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.908276+0000", 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0016043419999999999 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.908992+0000", 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071650700000000002 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.909074+0000", 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1862999999999999e-05 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.909131+0000", 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6839000000000002e-05 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:779 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.927402+0000", 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.404797759, 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004257767, 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 779 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927402+0000", 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927402+0000", 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927400+0000", 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927413+0000", 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3913999999999999e-05 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927444+0000", 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0636000000000003e-05 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927466+0000", 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1435999999999999e-05 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927584+0000", 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011885899999999999 2024-09-13T04:58:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927720+0000", 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013604599999999999 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.928301+0000", 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058032899999999998 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.931424+0000", 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031228889999999998 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.931502+0000", 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7948999999999995e-05 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.931593+0000", 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1413e-05 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.931660+0000", 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6600000000000006e-05 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:780 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.927528+0000", 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.4046717099999997, 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050481370000000003, 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 780 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927528+0000", 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927528+0000", 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927527+0000", 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927540+0000", 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3643e-05 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927566+0000", 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6233000000000001e-05 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.927591+0000", 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4606000000000001e-05 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.929077+0000", 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014862689999999999 2024-09-13T04:58:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.929396+0000", 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031825400000000002 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.930301+0000", 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00090494999999999996 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932293+0000", 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019920480000000002 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932396+0000", 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103046 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932474+0000", 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8713000000000003e-05 2024-09-13T04:58:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.932576+0000", 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101722 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:783 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.937892+0000", 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.39430751, 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035066630000000001, 2024-09-13T04:58:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 783 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.937892+0000", 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.937892+0000", 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.937890+0000", 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.937904+0000", 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3964e-05 2024-09-13T04:58:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.937943+0000", 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9696000000000001e-05 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.937962+0000", 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8170000000000001e-05 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.938053+0000", 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1186000000000004e-05 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.938244+0000", 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000191375 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.938862+0000", 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00061742800000000005 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.940265+0000", 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014033400000000001 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.941156+0000", 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00089139299999999998 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.941295+0000", 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013833899999999999 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.941399+0000", 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010408600000000001 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:785 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:09.965877+0000", 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.3663223159999998, 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043053199999999996, 2024-09-13T04:58:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 785 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.965877+0000", 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.965877+0000", 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.965874+0000", 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999967 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.965897+0000", 2024-09-13T04:58:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3637000000000001e-05 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.965937+0000", 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9085999999999998e-05 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.965957+0000", 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0247000000000001e-05 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.966057+0000", 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010011899999999999 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.966295+0000", 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023845499999999999 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.967425+0000", 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001129681 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.969567+0000", 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021424700000000001 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.970036+0000", 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00046823299999999997 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.970121+0000", 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5458999999999996e-05 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:09.970183+0000", 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1467e-05 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:831 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.055855+0000", 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.276345021, 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034657310000000001, 2024-09-13T04:58:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 831 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.055855+0000", 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.055855+0000", 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.055851+0000", 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999967 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.055874+0000", 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2558999999999999e-05 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.055915+0000", 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1056999999999997e-05 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.055935+0000", 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0157e-05 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.056082+0000", 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146785 2024-09-13T04:58:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.056374+0000", 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000291935 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.057151+0000", 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077711399999999997 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.058996+0000", 2024-09-13T04:58:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018452290000000001 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.059218+0000", 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000221945 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.059273+0000", 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4836000000000001e-05 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.059320+0000", 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7667000000000003e-05 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:841 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.080068+0000", 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.2521319880000004, 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034053199999999999, 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 841 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080068+0000", 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080068+0000", 2024-09-13T04:58:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080064+0000", 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999967 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080085+0000", 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1152999999999999e-05 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080127+0000", 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2169999999999998e-05 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080147+0000", 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9673999999999998e-05 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080293+0000", 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014576500000000001 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080470+0000", 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017738199999999999 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.080968+0000", 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049728299999999995 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.082730+0000", 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017628500000000001 2024-09-13T04:58:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.083312+0000", 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058181200000000002 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.083405+0000", 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2268999999999994e-05 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.083473+0000", 2024-09-13T04:58:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8459000000000003e-05 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:1130 3.16 3:6c409e0b:::rbd_object_map.112ac9124348:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.787914+0000", 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.544286048, 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033061079999999999, 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1130 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.787914+0000", 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.787914+0000", 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.787911+0000", 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.787925+0000", 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3603e-05 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.787955+0000", 2024-09-13T04:58:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0202999999999999e-05 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.787976+0000", 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1407e-05 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.788103+0000", 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126079 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.788268+0000", 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000165589 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.788926+0000", 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00065808800000000003 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.790479+0000", 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015524009999999999 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.791138+0000", 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00065924799999999997 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.791182+0000", 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4400000000000002e-05 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.791220+0000", 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7456000000000002e-05 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:1134 3.16 3:6b6d2fa2:::rbd_data.112ac9124348.000000000000000c:head [set-alloc-hint object_size 8388608 write_size 8388608,write 4917978~55166] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.801349+0000", 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5308508679999999, 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051958339999999999, 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1134 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801349+0000", 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801349+0000", 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801347+0000", 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801396+0000", 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8782000000000002e-05 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801415+0000", 2024-09-13T04:58:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8768000000000001e-05 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801424+0000", 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3039999999999994e-06 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801507+0000", 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2582000000000003e-05 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801649+0000", 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142142 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801912+0000", 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026283900000000001 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.806396+0000", 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044844220000000001 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.806409+0000", 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2602999999999999e-05 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.806485+0000", 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6274999999999997e-05 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.806545+0000", 2024-09-13T04:58:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9571000000000003e-05 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:1183 3.11 e30/24 3:8a4b9c83:::rbd_data.112ac9124348.0000000000000005:head v 30'22, mlcod=30'20)", 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.913358+0000", 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.418841478, 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004111951, 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.913358+0000", 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.913358+0000", 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.913357+0000", 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.913376+0000", 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8898000000000001e-05 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.913392+0000", 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5925000000000001e-05 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.913400+0000", 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1159999999999999e-06 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.913504+0000", 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010401800000000001 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.913544+0000", 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9986e-05 2024-09-13T04:58:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.917396+0000", 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038522069999999999 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.917470+0000", 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3813000000000006e-05 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:1195 3.e 3:71a0535c:::rbd_data.112ac9124348.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write 2015720~48869] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.933568+0000", 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.3986318889999998, 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043815929999999996, 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1195 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933568+0000", 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933568+0000", 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933566+0000", 2024-09-13T04:58:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933619+0000", 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2893000000000003e-05 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933640+0000", 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.128e-05 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933654+0000", 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.33e-05 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933701+0000", 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7419000000000001e-05 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933788+0000", 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7068000000000001e-05 2024-09-13T04:58:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.933936+0000", 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000148145 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.936745+0000", 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028086970000000002 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.937862+0000", 2024-09-13T04:58:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011165579999999999 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.937902+0000", 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0467999999999999e-05 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.937949+0000", 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7247999999999999e-05 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:7683 3.16 3:68446a75:::rbd_data.112a2c87bb71.0000000000000140:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:14.271450+0000", 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.0607500519999999, 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047166630000000003, 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 7683 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.271450+0000", 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.271450+0000", 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.271447+0000", 2024-09-13T04:58:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.271457+0000", 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.271483+0000", 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6013e-05 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.271493+0000", 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4730000000000006e-06 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.272088+0000", 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00059503300000000002 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.272182+0000", 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4389999999999996e-05 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.272282+0000", 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100215 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.273449+0000", 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011667100000000001 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.276089+0000", 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002640356 2024-09-13T04:58:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.276131+0000", 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1690000000000002e-05 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:14.276166+0000", 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5277000000000001e-05 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:13761 3.5 3:a7a9a0ab:::rbd_data.112a2c87bb71.0000000000000118:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.741580+0000", 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.59061985100000003, 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041876580000000004, 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13761 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741580+0000", 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741580+0000", 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741578+0000", 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741586+0000", 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.943e-06 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741603+0000", 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6971000000000001e-05 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741611+0000", 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9829999999999996e-06 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741657+0000", 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6029999999999998e-05 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741729+0000", 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2360000000000005e-05 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.741878+0000", 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014897399999999999 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.742873+0000", 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00099468100000000008 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.745680+0000", 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028073130000000001 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.745730+0000", 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.994e-05 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.745767+0000", 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7311000000000002e-05 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:13769 3.5 3:a55a4cd2:::rbd_data.112a2c87bb71.0000000000000120:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.58839337700000005, 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007083972, 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13769 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743805+0000", 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743810+0000", 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0540000000000002e-06 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743823+0000", 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.275e-05 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743831+0000", 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6750000000000002e-06 2024-09-13T04:58:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743877+0000", 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6597000000000001e-05 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743956+0000", 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8732000000000004e-05 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.744121+0000", 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016536900000000001 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.745356+0000", 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001234824 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750813+0000", 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00545671 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750854+0000", 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1309e-05 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750890+0000", 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5927999999999999e-05 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:13779 3.5 3:a05d5a5a:::rbd_data.112a2c87bb71.0000000000000134:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.747894+0000", 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.58430579100000002, 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004330647, 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13779 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.747894+0000", 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.747894+0000", 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.747893+0000", 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.747898+0000", 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4759999999999997e-06 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.747913+0000", 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4617000000000001e-05 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.747935+0000", 2024-09-13T04:58:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2178999999999999e-05 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.747984+0000", 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8368000000000002e-05 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.748045+0000", 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1205999999999996e-05 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.748210+0000", 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016521999999999999 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.749120+0000", 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000909862 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.752149+0000", 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030288110000000002 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.752191+0000", 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2216999999999999e-05 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.752224+0000", 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3618999999999998e-05 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:15085 3.11 e35/24 3:8ba33952:::rbd_data.112a2c87bb71.0000000000000021:head v 35'52, mlcod=35'49)", 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.843874+0000", 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.488325597, 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035117669999999998, 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.843874+0000", 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.843874+0000", 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.843872+0000", 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.844112+0000", 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023987699999999999 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.844133+0000", 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0417000000000001e-05 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.844141+0000", 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4929999999999998e-06 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.844201+0000", 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9654000000000002e-05 2024-09-13T04:58:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.844227+0000", 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5950999999999999e-05 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.847339+0000", 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031127949999999998 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.847386+0000", 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6383e-05 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:15109 3.2 e35/24 3:43e66e79:::rbd_data.112a2c87bb71.0000000000000027:head v 35'10, mlcod=34'8)", 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.889374+0000", 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.44282597200000001, 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043468889999999996, 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.889374+0000", 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.889374+0000", 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.889372+0000", 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.890037+0000", 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00066443100000000005 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.890052+0000", 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5412000000000001e-05 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.890059+0000", 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3860000000000001e-06 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.890101+0000", 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1570999999999999e-05 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.890116+0000", 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4841000000000001e-05 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.893682+0000", 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035664680000000002 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.893721+0000", 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8217999999999999e-05 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:15388 3.16 3:6f27987b:::rbd_data.112a2c87bb71.0000000000000121:7 [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.016610+0000", 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.31558930899999998, 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004319799, 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 15388 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.016610+0000", 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.016610+0000", 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.016610+0000", 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.016616+0000", 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2230000000000004e-06 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.016627+0000", 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1341e-05 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.016638+0000", 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1327e-05 2024-09-13T04:58:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.020747+0000", 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041087210000000001 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.020835+0000", 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8247999999999999e-05 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.020930+0000", 2024-09-13T04:58:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4753000000000006e-05 2024-09-13T04:58:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.449 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:16.450 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:16.451 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:16.448+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:16.506 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:16.507 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:16.508 INFO:teuthology.orchestra.run.smithi001.stdout:770 writesame 0x870fe14 thru 0x87109da data_size 0xf(0xbc7 bytes) 2024-09-13T04:58:16.540 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:16.551 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:16.548+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:16.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:16.600 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:16.617 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:16.617 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:16.637 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:16.638 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:16.652 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:16.652+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:16.653 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:16.654 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:16.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:16.672 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:16.673 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 35, 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "8850da77-4384-4335-9559-af2e805c4f72", 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-13T04:57:34.164121+0000", 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-13T04:58:15.269931+0000", 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-13T04:57:47.098606+0000", 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-13T04:57:36.815703+0000", 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5801984, 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "noscrub", 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-13T04:58:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "jewel", 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "reef", 2024-09-13T04:58:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:48.984599+0000", 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "19", 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-13T04:58:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:52.756781+0000", 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-13T04:58:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "23", 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 3, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 23, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 4.9800000190734863, 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 4.9800000190734863, 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 4.380000114440918, 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 4.380000114440918, 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:58:02.446076+0000", 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "35", 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 7, 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 35, 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-13T04:58:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "2fda29ca-3c6d-47ae-9244-8ccdf55f89f6", 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6820/2346968854", 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6824/2346968854", 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/2346968854", 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/2346968854", 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9ca6e867-faf0-4295-b8eb-4c738dd26cb9", 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-13T04:58:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/466532899", 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6821/466532899", 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/466532899", 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/466532899", 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d8159400-8662-4567-8714-b48c2d8f3ad2", 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/343712578", 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/343712578", 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/343712578", 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/343712578", 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "13a3ea70-fa37-41d8-bae7-0203952b02cb", 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/2021827348", 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/2021827348", 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/2021827348", 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/2021827348", 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e896e6b5-c54a-405f-ab69-2a5b7b7a153e", 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6800", 2024-09-13T04:58:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6801", 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6802", 2024-09-13T04:58:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6803", 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6806", 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6807", 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6804", 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6805", 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6801/2183053562", 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6803/2183053562", 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6807/2183053562", 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6805/2183053562", 2024-09-13T04:58:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7de23072-7eda-43d6-9b92-d608ef82c6fa", 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6816", 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6817", 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6819", 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6821", 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6826", 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6828", 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6822", 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6824", 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6817/2765053006", 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6821/2765053006", 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6828/2765053006", 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6824/2765053006", 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8719fad9-81a0-46be-a546-db83bcace33b", 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6818", 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6820", 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6823", 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6825", 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6830", 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6831", 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6827", 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6829", 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6820/2721913793", 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6825/2721913793", 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6831/2721913793", 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6829/2721913793", 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8bf0a351-e58e-4393-a6ce-fe70acf362aa", 2024-09-13T04:58:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6808", 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6809", 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6810", 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6811", 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6814", 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6815", 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6812", 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6813", 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6809/4202513770", 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6811/4202513770", 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6815/4202513770", 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6813/4202513770", 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:58:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.092681+0000", 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.414226+0000", 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.368701+0000", 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.370707+0000", 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.980085+0000", 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:58:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.874193+0000", 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.219724+0000", 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.114987+0000", 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-13T04:58:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-13T04:58:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-13T04:58:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-13T04:58:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:16.765 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:16.766 INFO:teuthology.orchestra.run.smithi001.stdout:771 write 0xbca1ef8 thru 0xbca597d (0x3a86 bytes) 2024-09-13T04:58:16.766 INFO:teuthology.orchestra.run.smithi001.stdout:772 punch from 0x9ab4a6d to 0x9ac379c, (0xed2f bytes) 2024-09-13T04:58:16.766 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:16.766 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:16.767 INFO:teuthology.orchestra.run.smithi001.stdout:773 writesame 0xe4b32b9 thru 0xe4b7853 data_size 0x1(0x459b bytes) 2024-09-13T04:58:16.768 INFO:tasks.thrashosds.thrasher:j is {'epoch': 35, 'fsid': '8850da77-4384-4335-9559-af2e805c4f72', 'created': '2024-09-13T04:57:34.164121+0000', 'modified': '2024-09-13T04:58:15.269931+0000', 'last_up_change': '2024-09-13T04:57:47.098606+0000', 'last_in_change': '2024-09-13T04:57:36.815703+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T04:57:48.984599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T04:57:52.756781+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T04:58:02.446076+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 7, 'snap_epoch': 35, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2fda29ca-3c6d-47ae-9244-8ccdf55f89f6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6820', 'nonce': 2346968854}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6824', 'nonce': 2346968854}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 2346968854}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 2346968854}]}, 'public_addr': '172.21.15.1:6820/2346968854', 'cluster_addr': '172.21.15.1:6824/2346968854', 'heartbeat_back_addr': '172.21.15.1:6831/2346968854', 'heartbeat_front_addr': '172.21.15.1:6828/2346968854', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9ca6e867-faf0-4295-b8eb-4c738dd26cb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 466532899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6819', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 466532899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 466532899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6823', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 466532899}]}, 'public_addr': '172.21.15.1:6817/466532899', 'cluster_addr': '172.21.15.1:6821/466532899', 'heartbeat_back_addr': '172.21.15.1:6829/466532899', 'heartbeat_front_addr': '172.21.15.1:6825/466532899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8159400-8662-4567-8714-b48c2d8f3ad2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 343712578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 343712578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 343712578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 343712578}]}, 'public_addr': '172.21.15.1:6801/343712578', 'cluster_addr': '172.21.15.1:6803/343712578', 'heartbeat_back_addr': '172.21.15.1:6807/343712578', 'heartbeat_front_addr': '172.21.15.1:6805/343712578', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '13a3ea70-fa37-41d8-bae7-0203952b02cb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 2021827348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 2021827348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 2021827348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 2021827348}]}, 'public_addr': '172.21.15.1:6809/2021827348', 'cluster_addr': '172.21.15.1:6811/2021827348', 'heartbeat_back_addr': '172.21.15.1:6815/2021827348', 'heartbeat_front_addr': '172.21.15.1:6813/2021827348', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e896e6b5-c54a-405f-ab69-2a5b7b7a153e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 2183053562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 2183053562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 2183053562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 2183053562}]}, 'public_addr': '172.21.15.144:6801/2183053562', 'cluster_addr': '172.21.15.144:6803/2183053562', 'heartbeat_back_addr': '172.21.15.144:6807/2183053562', 'heartbeat_front_addr': '172.21.15.144:6805/2183053562', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7de23072-7eda-43d6-9b92-d608ef82c6fa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6816', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2765053006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 2765053006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6826', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6828', 'nonce': 2765053006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6824', 'nonce': 2765053006}]}, 'public_addr': '172.21.15.144:6817/2765053006', 'cluster_addr': '172.21.15.144:6821/2765053006', 'heartbeat_back_addr': '172.21.15.144:6828/2765053006', 'heartbeat_front_addr': '172.21.15.144:6824/2765053006', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8719fad9-81a0-46be-a546-db83bcace33b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2721913793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6823', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6825', 'nonce': 2721913793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6830', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6831', 'nonce': 2721913793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6827', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6829', 'nonce': 2721913793}]}, 'public_addr': '172.21.15.144:6820/2721913793', 'cluster_addr': '172.21.15.144:6825/2721913793', 'heartbeat_back_addr': '172.21.15.144:6831/2721913793', 'heartbeat_front_addr': '172.21.15.144:6829/2721913793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8bf0a351-e58e-4393-a6ce-fe70acf362aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6809', 'nonce': 4202513770}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6810', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 4202513770}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6815', 'nonce': 4202513770}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 4202513770}]}, 'public_addr': '172.21.15.144:6809/4202513770', 'cluster_addr': '172.21.15.144:6811/4202513770', 'heartbeat_back_addr': '172.21.15.144:6815/4202513770', 'heartbeat_front_addr': '172.21.15.144:6813/4202513770', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.092681+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.414226+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.368701+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.370707+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.980085+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.874193+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.219724+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.114987+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T04:58:16.768 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:58:16.769 INFO:teuthology.orchestra.run.smithi001.stdout:775 punch from 0x9554a94 to 0x955fef5, (0xb461 bytes) 2024-09-13T04:58:16.769 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:16.769 INFO:teuthology.orchestra.run.smithi001.stdout:778 trunc from 0xe4b7854 to 0xe2738e9 2024-09-13T04:58:16.769 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:16.770 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:16.770 INFO:teuthology.orchestra.run.smithi001.stdout:781 writesame 0x9bf5c10 thru 0x9c02625 data_size 0x1(0xca16 bytes) 2024-09-13T04:58:16.771 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:16.764+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:16.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:16.905 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:16.904+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:16.968 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:16.968+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:17.032 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:17.033 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:17.045 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:17.076 INFO:teuthology.orchestra.run.smithi001.stdout:783 read 0xbe7295b thru 0xbe77502 (0x4ba8 bytes) 2024-09-13T04:58:17.076 INFO:teuthology.orchestra.run.smithi001.stdout:785 writesame 0x28a337 thru 0x29303f data_size 0x1d(0x8d09 bytes) 2024-09-13T04:58:17.080 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:17.080 INFO:teuthology.orchestra.run.smithi001.stdout:791 punch from 0xc4fd275 to 0xc50aa1a, (0xd7a5 bytes) 2024-09-13T04:58:17.092 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:17.092+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:17.101 INFO:teuthology.orchestra.run.smithi001.stdout:792 trunc from 0xe2738e9 to 0x24a2ae7 2024-09-13T04:58:17.154 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:17.155 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:58:17.169 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:17.168+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:17.172 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-09-13T04:58:16.914048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107170616,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":13,"num_read":4340,"num_read_kb":4510,"num_write":1201,"num_write_kb":8233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":977,"ondisk_log_size":977,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249244,"kb_used_data":4368,"kb_used_omap":12,"kb_used_meta":244723,"kb_avail":754725476,"statfs":{"total":773094113280,"available":772838887424,"internally_reserved":0,"allocated":4472832,"data_stored":4277087,"data_compressed":914261,"data_compressed_allocated":1089536,"data_compressed_original":2899968,"omap_allocated":12743,"internal_metadata":250596921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67144356,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":9,"num_read":3917,"num_read_kb":3882,"num_write":930,"num_write_kb":4682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.653836"},"pg_stats":[{"pgid":"3.1f","version":"31'37","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.856359+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:58:11.856359+0000","last_peered":"2024-09-13T04:58:11.856359+0000","last_clean":"2024-09-13T04:58:11.856359+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:58:11.856359+0000","last_undegraded":"2024-09-13T04:58:11.856359+0000","last_fullsized":"2024-09-13T04:58:11.856359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":12582968,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":86,"num_read_kb":343,"num_write":36,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T04:58:10.283200+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:58:10.283200+0000","last_peered":"2024-09-13T04:58:10.283200+0000","last_clean":"2024-09-13T04:58:10.283200+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:58:10.283200+0000","last_undegraded":"2024-09-13T04:58:10.283200+0000","last_fullsized":"2024-09-13T04:58:10.283200+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"31'25","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.105244+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:58:12.105244+0000","last_peered":"2024-09-13T04:58:12.105244+0000","last_clean":"2024-09-13T04:58:12.105244+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:58:12.105244+0000","last_undegraded":"2024-09-13T04:58:12.105244+0000","last_fullsized":"2024-09-13T04:58:12.105244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":11540858,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":17,"num_read_kb":197,"num_write":24,"num_write_kb":460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"32'16","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.455346+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:58:12.455346+0000","last_peered":"2024-09-13T04:58:12.455346+0000","last_clean":"2024-09-13T04:58:12.455346+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:58:12.455346+0000","last_undegraded":"2024-09-13T04:58:12.455346+0000","last_fullsized":"2024-09-13T04:58:12.455346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":16,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"32'17","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.537445+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:58:12.537445+0000","last_peered":"2024-09-13T04:58:12.537445+0000","last_clean":"2024-09-13T04:58:12.537445+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:58:12.537445+0000","last_undegraded":"2024-09-13T04:58:12.537445+0000","last_fullsized":"2024-09-13T04:58:12.537445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":32,"num_write":18,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"32'20","reported_seq":179,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.606739+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:58:12.606739+0000","last_peered":"2024-09-13T04:58:12.606739+0000","last_clean":"2024-09-13T04:58:12.606739+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:58:12.606739+0000","last_undegraded":"2024-09-13T04:58:12.606739+0000","last_fullsized":"2024-09-13T04:58:12.606739+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":6830680,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":152,"num_read_kb":219,"num_write":20,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"32'25","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.592291+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:58:12.592291+0000","last_peered":"2024-09-13T04:58:12.592291+0000","last_clean":"2024-09-13T04:58:12.592291+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:58:12.592291+0000","last_undegraded":"2024-09-13T04:58:12.592291+0000","last_fullsized":"2024-09-13T04:58:12.592291+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":3380112,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":39,"num_write":24,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"31'31","reported_seq":85,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.851591+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:58:11.851591+0000","last_peered":"2024-09-13T04:58:11.851591+0000","last_clean":"2024-09-13T04:58:11.851591+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:58:11.851591+0000","last_undegraded":"2024-09-13T04:58:11.851591+0000","last_fullsized":"2024-09-13T04:58:11.851591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":14680107,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":70,"num_read_kb":67,"num_write":30,"num_write_kb":177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"31'3","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.826876+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:58:11.826876+0000","last_peered":"2024-09-13T04:58:11.826876+0000","last_clean":"2024-09-13T04:58:11.826876+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:58:11.826876+0000","last_undegraded":"2024-09-13T04:58:11.826876+0000","last_fullsized":"2024-09-13T04:58:11.826876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":3376261,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"31'206","reported_seq":230,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.788683+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:58:11.788683+0000","last_peered":"2024-09-13T04:58:11.788683+0000","last_clean":"2024-09-13T04:58:11.788683+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:58:11.788683+0000","last_undegraded":"2024-09-13T04:58:11.788683+0000","last_fullsized":"2024-09-13T04:58:11.788683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":1048606,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":949,"num_read_kb":749,"num_write":325,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"32'4","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.307754+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:58:12.307754+0000","last_peered":"2024-09-13T04:58:12.307754+0000","last_clean":"2024-09-13T04:58:12.307754+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:58:12.307754+0000","last_undegraded":"2024-09-13T04:58:12.307754+0000","last_fullsized":"2024-09-13T04:58:12.307754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"32'4","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.477885+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:58:12.477885+0000","last_peered":"2024-09-13T04:58:12.477885+0000","last_clean":"2024-09-13T04:58:12.477885+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:58:12.477885+0000","last_undegraded":"2024-09-13T04:58:12.477885+0000","last_fullsized":"2024-09-13T04:58:12.477885+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":5,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252694+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:58:11.252694+0000","last_peered":"2024-09-13T04:58:11.252694+0000","last_clean":"2024-09-13T04:58:11.252694+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:58:11.252694+0000","last_undegraded":"2024-09-13T04:58:11.252694+0000","last_fullsized":"2024-09-13T04:58:11.252694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'2","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.317984+0000","last_change":"2024-09-13T04:58:05.912872+0000","last_active":"2024-09-13T04:58:12.317984+0000","last_peered":"2024-09-13T04:58:12.317984+0000","last_clean":"2024-09-13T04:58:12.317984+0000","last_became_active":"2024-09-13T04:58:03.926978+0000","last_became_peered":"2024-09-13T04:58:03.926978+0000","last_unstale":"2024-09-13T04:58:12.317984+0000","last_undegraded":"2024-09-13T04:58:12.317984+0000","last_fullsized":"2024-09-13T04:58:12.317984+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:30:43.308903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.250466+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:58:11.250466+0000","last_peered":"2024-09-13T04:58:11.250466+0000","last_clean":"2024-09-13T04:58:11.250466+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:58:11.250466+0000","last_undegraded":"2024-09-13T04:58:11.250466+0000","last_fullsized":"2024-09-13T04:58:11.250466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"32'18","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.317262+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:58:12.317262+0000","last_peered":"2024-09-13T04:58:12.317262+0000","last_clean":"2024-09-13T04:58:12.317262+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:58:12.317262+0000","last_undegraded":"2024-09-13T04:58:12.317262+0000","last_fullsized":"2024-09-13T04:58:12.317262+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":18,"num_write_kb":178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252550+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:58:11.252550+0000","last_peered":"2024-09-13T04:58:11.252550+0000","last_clean":"2024-09-13T04:58:11.252550+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:58:11.252550+0000","last_undegraded":"2024-09-13T04:58:11.252550+0000","last_fullsized":"2024-09-13T04:58:11.252550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"32'5","reported_seq":25,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.396756+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:58:12.396756+0000","last_peered":"2024-09-13T04:58:12.396756+0000","last_clean":"2024-09-13T04:58:12.396756+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:58:12.396756+0000","last_undegraded":"2024-09-13T04:58:12.396756+0000","last_fullsized":"2024-09-13T04:58:12.396756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T04:58:10.283182+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:58:10.283182+0000","last_peered":"2024-09-13T04:58:10.283182+0000","last_clean":"2024-09-13T04:58:10.283182+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:58:10.283182+0000","last_undegraded":"2024-09-13T04:58:10.283182+0000","last_fullsized":"2024-09-13T04:58:10.283182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"30'14","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.249558+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:58:11.249558+0000","last_peered":"2024-09-13T04:58:11.249558+0000","last_clean":"2024-09-13T04:58:11.249558+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:58:11.249558+0000","last_undegraded":"2024-09-13T04:58:11.249558+0000","last_fullsized":"2024-09-13T04:58:11.249558+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":14,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"32'26","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.598971+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:58:12.598971+0000","last_peered":"2024-09-13T04:58:12.598971+0000","last_clean":"2024-09-13T04:58:12.598971+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:58:12.598971+0000","last_undegraded":"2024-09-13T04:58:12.598971+0000","last_fullsized":"2024-09-13T04:58:12.598971+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":6232992,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":4,"num_read_kb":0,"num_write":25,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252384+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:58:11.252384+0000","last_peered":"2024-09-13T04:58:11.252384+0000","last_clean":"2024-09-13T04:58:11.252384+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:58:11.252384+0000","last_undegraded":"2024-09-13T04:58:11.252384+0000","last_fullsized":"2024-09-13T04:58:11.252384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"31'16","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.373080+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:58:11.373080+0000","last_peered":"2024-09-13T04:58:11.373080+0000","last_clean":"2024-09-13T04:58:11.373080+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:58:11.373080+0000","last_undegraded":"2024-09-13T04:58:11.373080+0000","last_fullsized":"2024-09-13T04:58:11.373080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":16,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.310012+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:58:11.310012+0000","last_peered":"2024-09-13T04:58:11.310012+0000","last_clean":"2024-09-13T04:58:11.310012+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:58:11.310012+0000","last_undegraded":"2024-09-13T04:58:11.310012+0000","last_fullsized":"2024-09-13T04:58:11.310012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"31'23","reported_seq":73,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.843783+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:58:11.843783+0000","last_peered":"2024-09-13T04:58:11.843783+0000","last_clean":"2024-09-13T04:58:11.843783+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:58:11.843783+0000","last_undegraded":"2024-09-13T04:58:11.843783+0000","last_fullsized":"2024-09-13T04:58:11.843783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":7636017,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":46,"num_read_kb":264,"num_write":22,"num_write_kb":374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252077+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:58:11.252077+0000","last_peered":"2024-09-13T04:58:11.252077+0000","last_clean":"2024-09-13T04:58:11.252077+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:58:11.252077+0000","last_undegraded":"2024-09-13T04:58:11.252077+0000","last_fullsized":"2024-09-13T04:58:11.252077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"28'1","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.250473+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:58:11.250473+0000","last_peered":"2024-09-13T04:58:11.250473+0000","last_clean":"2024-09-13T04:58:11.250473+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:58:11.250473+0000","last_undegraded":"2024-09-13T04:58:11.250473+0000","last_fullsized":"2024-09-13T04:58:11.250473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.252377+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:58:11.252377+0000","last_peered":"2024-09-13T04:58:11.252377+0000","last_clean":"2024-09-13T04:58:11.252377+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:58:11.252377+0000","last_undegraded":"2024-09-13T04:58:11.252377+0000","last_fullsized":"2024-09-13T04:58:11.252377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.248127+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:58:11.248127+0000","last_peered":"2024-09-13T04:58:11.248127+0000","last_clean":"2024-09-13T04:58:11.248127+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:58:11.248127+0000","last_undegraded":"2024-09-13T04:58:11.248127+0000","last_fullsized":"2024-09-13T04:58:11.248127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"31'13","reported_seq":142,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.616682+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:58:11.616682+0000","last_peered":"2024-09-13T04:58:11.616682+0000","last_clean":"2024-09-13T04:58:11.616682+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:58:11.616682+0000","last_undegraded":"2024-09-13T04:58:11.616682+0000","last_fullsized":"2024-09-13T04:58:11.616682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":330,"num_read_kb":284,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"32'108","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.627004+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:58:12.627004+0000","last_peered":"2024-09-13T04:58:12.627004+0000","last_clean":"2024-09-13T04:58:12.627004+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:58:12.627004+0000","last_undegraded":"2024-09-13T04:58:12.627004+0000","last_fullsized":"2024-09-13T04:58:12.627004+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":4162346,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":468,"num_read_kb":371,"num_write":172,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"32'26","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.477909+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:58:12.477909+0000","last_peered":"2024-09-13T04:58:12.477909+0000","last_clean":"2024-09-13T04:58:12.477909+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:58:12.477909+0000","last_undegraded":"2024-09-13T04:58:12.477909+0000","last_fullsized":"2024-09-13T04:58:12.477909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":27,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.310045+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:58:11.310045+0000","last_peered":"2024-09-13T04:58:11.310045+0000","last_clean":"2024-09-13T04:58:11.310045+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:58:11.310045+0000","last_undegraded":"2024-09-13T04:58:11.310045+0000","last_fullsized":"2024-09-13T04:58:11.310045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"28'11","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.306081+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:58:11.306081+0000","last_peered":"2024-09-13T04:58:11.306081+0000","last_clean":"2024-09-13T04:58:11.306081+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:58:11.306081+0000","last_undegraded":"2024-09-13T04:58:11.306081+0000","last_fullsized":"2024-09-13T04:58:11.306081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":43,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":31,"num_write":15,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"31'103","reported_seq":636,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.789549+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:58:11.789549+0000","last_peered":"2024-09-13T04:58:11.789549+0000","last_clean":"2024-09-13T04:58:11.789549+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:58:11.789549+0000","last_undegraded":"2024-09-13T04:58:11.789549+0000","last_fullsized":"2024-09-13T04:58:11.789549+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":2413040,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":1420,"num_read_kb":1256,"num_write":106,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"31'15","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T04:58:11.355115+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:58:11.355115+0000","last_peered":"2024-09-13T04:58:11.355115+0000","last_clean":"2024-09-13T04:58:11.355115+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:58:11.355115+0000","last_undegraded":"2024-09-13T04:58:11.355115+0000","last_fullsized":"2024-09-13T04:58:11.355115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":4492964,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":14,"num_write_kb":198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"32'8","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.534988+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:58:12.534988+0000","last_peered":"2024-09-13T04:58:12.534988+0000","last_clean":"2024-09-13T04:58:12.534988+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:58:12.534988+0000","last_undegraded":"2024-09-13T04:58:12.534988+0000","last_fullsized":"2024-09-13T04:58:12.534988+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":3145728,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"32'10","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.537525+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:58:12.537525+0000","last_peered":"2024-09-13T04:58:12.537525+0000","last_clean":"2024-09-13T04:58:12.537525+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:58:12.537525+0000","last_undegraded":"2024-09-13T04:58:12.537525+0000","last_fullsized":"2024-09-13T04:58:12.537525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":3145728,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":28,"num_write":10,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"32'30","reported_seq":68,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.495086+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:58:12.495086+0000","last_peered":"2024-09-13T04:58:12.495086+0000","last_clean":"2024-09-13T04:58:12.495086+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:58:12.495086+0000","last_undegraded":"2024-09-13T04:58:12.495086+0000","last_fullsized":"2024-09-13T04:58:12.495086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":9396671,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":42,"num_read_kb":52,"num_write":30,"num_write_kb":439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"32'7","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.515926+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:58:12.515926+0000","last_peered":"2024-09-13T04:58:12.515926+0000","last_clean":"2024-09-13T04:58:12.515926+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:58:12.515926+0000","last_undegraded":"2024-09-13T04:58:12.515926+0000","last_fullsized":"2024-09-13T04:58:12.515926+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":36,"num_write":8,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"32'33","reported_seq":234,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T04:58:12.641714+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:58:12.641714+0000","last_peered":"2024-09-13T04:58:12.641714+0000","last_clean":"2024-09-13T04:58:12.641714+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:58:12.641714+0000","last_undegraded":"2024-09-13T04:58:12.641714+0000","last_fullsized":"2024-09-13T04:58:12.641714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":3077892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":483,"num_read_kb":422,"num_write":33,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":106580229,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":13,"num_read":4224,"num_read_kb":4410,"num_write":1040,"num_write_kb":5815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1830912,"data_stored":2249236,"data_compressed":1015913,"data_compressed_allocated":1146880,"data_compressed_original":1925120,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":127535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":374306,"data_compressed":124778,"data_compressed_allocated":135168,"data_compressed_original":196608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":413696,"data_stored":218155,"data_compressed":40766,"data_compressed_allocated":49152,"data_compressed_original":77824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":737280,"data_stored":622226,"data_compressed":200816,"data_compressed_allocated":225280,"data_compressed_original":409600,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35768,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94336072,"statfs":{"total":96636764160,"available":96600137728,"internally_reserved":0,"allocated":569344,"data_stored":394413,"data_compressed":149112,"data_compressed_allocated":155648,"data_compressed_original":225280,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":696320,"data_stored":1134811,"data_compressed":163978,"data_compressed_allocated":221184,"data_compressed_original":933888,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":528384,"data_stored":402170,"data_compressed":103360,"data_compressed_allocated":118784,"data_compressed_original":237568,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":626688,"data_stored":1003471,"data_compressed":131451,"data_compressed_allocated":184320,"data_compressed_original":819200,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":196861,"data_compressed":100496,"data_compressed_allocated":110592,"data_compressed_original":167936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":452694,"data_compressed":179459,"data_compressed_allocated":208896,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":310099,"data_compressed":179010,"data_compressed_allocated":188416,"data_compressed_original":270336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":507366,"data_compressed":190178,"data_compressed_allocated":221184,"data_compressed_original":430080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":160942,"data_compressed":84153,"data_compressed_allocated":98304,"data_compressed_original":143360,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":380213,"data_compressed":157839,"data_compressed_allocated":184320,"data_compressed_original":323584,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":241002,"data_compressed":124778,"data_compressed_allocated":135168,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":59,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:58:17.174 INFO:tasks.thrashosds.thrasher:Setting 3.6 to [6, 4] 2024-09-13T04:58:17.174 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.6', '6', '4'] 2024-09-13T04:58:17.175 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.6 6 4 2024-09-13T04:58:17.208 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:17.208 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:17.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:17.270 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:17.268+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:17.276 INFO:teuthology.orchestra.run.smithi001.stdout:793 writesame 0x698424c thru 0x6985e73 data_size 0x35(0x1c28 bytes) 2024-09-13T04:58:17.277 INFO:teuthology.orchestra.run.smithi001.stdout:796 read 0x2b43918 thru 0x2b48b9f (0x5288 bytes) 2024-09-13T04:58:17.278 INFO:teuthology.orchestra.run.smithi001.stdout:798 read 0x605034f thru 0x60552f8 (0x4faa bytes) 2024-09-13T04:58:17.278 INFO:teuthology.orchestra.run.smithi001.stdout:799 clone 5 order 19 su 16384 sc 2 2024-09-13T04:58:17.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:58:17.371 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:17.368+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:17.383 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:17.383 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:17.397 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:17.472 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:17.468+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:17.519 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:17.519 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:17.533 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:17.573 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:17.572+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:17.655 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:17.655 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:17.669 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:17.716 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:17.716+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:17.774 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:17.772+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:17.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:17.892 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:17.892+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:17.970 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:17.970 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:17.976 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:17.976+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:17.983 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:18.077 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:18.076+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:18.146 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:18.147 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:18.161 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:18.178 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:18.176+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:18.279 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:18.276+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:18.285 INFO:teuthology.orchestra.run.smithi001.stderr:set 3.6 pg_upmap mapping to [6,4] 2024-09-13T04:58:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:58:18.333 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:18.333 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:18.348 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:18.379 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:18.376+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:18.416 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone4 from 0x6985e74 (overlap 0x24a2ae7) to 0x3a9de3 2024-09-13T04:58:18.469 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:18.469 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:18.481 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:18.480+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:18.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:18.582 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:18.580+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:18.645 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:18.645 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:18.659 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:18.684 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:18.680+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:18.771 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:18.817 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:18.816+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:18.821 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:18.821 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:18.835 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:18.886 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:18.884+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:18.932 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:18.933 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:18.949 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:18.949 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:18.959 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:18.959 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:18.965 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:18.965 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:18.974 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:18.981 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:18.982 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:18.987 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:18.984+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:18.999 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:18.999 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:19.019 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:19.019 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:19.037 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:19.038 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:19.058 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:19.058 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:19.088 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:19.088+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:19.090 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:19.090 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:19.090 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:19.091 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:19.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:19.190 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:19.188+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:19.273 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:19.273 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:19.273 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:19.273 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:19.273 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:19.288 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:19.290 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:19.288+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:19.391 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:19.388+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:19.413 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:19.413 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:19.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:19.413 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:14863 3.f 3:f3429905:::rbd_data.112af6c12117.0000000000000003:6 [read 2039808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.771956+0000", 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.639269401, 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037867816999999998, 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 14863 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.414 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.771956+0000", 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.771956+0000", 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.771955+0000", 2024-09-13T04:58:19.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.771962+0000", 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5549999999999999e-06 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.771968+0000", 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7679999999999997e-06 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.771972+0000", 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.369e-06 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.807782+0000", 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035810054000000001 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.807859+0000", 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6769999999999999e-05 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.809823+0000", 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019646020000000002 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17073 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4038656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076819+0000", 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3344058099999998, 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036663172000000001, 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17073 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076819+0000", 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076819+0000", 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076819+0000", 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076821+0000", 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0480000000000001e-06 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076828+0000", 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8050000000000001e-06 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076832+0000", 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0090000000000001e-06 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113364+0000", 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.03653265 2024-09-13T04:58:19.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113431+0000", 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6561000000000002e-05 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113482+0000", 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1409000000000002e-05 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17074 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3592192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076836+0000", 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3343893200000001, 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036777284, 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17074 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076836+0000", 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076836+0000", 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076835+0000", 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076841+0000", 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8259999999999998e-06 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076849+0000", 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5780000000000001e-06 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076854+0000", 2024-09-13T04:58:19.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7650000000000001e-06 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113510+0000", 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036656082 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113570+0000", 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0427000000000003e-05 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113613+0000", 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2925e-05 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17075 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4079616~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076858+0000", 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3343673630000001, 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036888252000000003, 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17075 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076858+0000", 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076858+0000", 2024-09-13T04:58:19.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076857+0000", 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076859+0000", 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1569999999999998e-06 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076865+0000", 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7559999999999996e-06 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076870+0000", 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7099999999999998e-06 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113638+0000", 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036768575999999997 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113677+0000", 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8954e-05 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113746+0000", 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8428000000000003e-05 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17076 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4120576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.431 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076874+0000", 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3343511010000002, 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036972953000000003, 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17076 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076874+0000", 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076874+0000", 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076874+0000", 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076876+0000", 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0990000000000001e-06 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076883+0000", 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076887+0000", 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3340000000000003e-06 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113769+0000", 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036882218000000001 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113808+0000", 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8590000000000002e-05 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113847+0000", 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8893999999999997e-05 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17077 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3633152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076892+0000", 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3343331799999998, 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037099542999999999, 2024-09-13T04:58:19.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17077 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076892+0000", 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076892+0000", 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076891+0000", 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076894+0000", 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3120000000000001e-06 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076901+0000", 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2989999999999999e-06 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076906+0000", 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.955e-06 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113872+0000", 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036966273000000001 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113945+0000", 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2779000000000002e-05 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.113991+0000", 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6267000000000003e-05 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.440 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17078 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4161536~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076910+0000", 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3343150220000002, 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037205556000000001, 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17078 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076910+0000", 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076910+0000", 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076910+0000", 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076912+0000", 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0420000000000001e-06 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076919+0000", 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0820000000000001e-06 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076923+0000", 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5979999999999999e-06 2024-09-13T04:58:19.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114017+0000", 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037094031 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114075+0000", 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7436999999999999e-05 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114116+0000", 2024-09-13T04:58:19.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.066e-05 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17079 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3674112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076928+0000", 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3342965410000001, 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037338757, 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17079 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076928+0000", 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076928+0000", 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076928+0000", 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076930+0000", 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.215e-06 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076937+0000", 2024-09-13T04:58:19.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1199999999999996e-06 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076942+0000", 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6589999999999998e-06 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114139+0000", 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037196626000000003 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114177+0000", 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8087000000000003e-05 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114267+0000", 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0340000000000006e-05 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17080 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3715072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076946+0000", 2024-09-13T04:58:19.449 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3342785340000001, 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037442877999999999, 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17080 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076946+0000", 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076946+0000", 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076946+0000", 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076948+0000", 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0389999999999999e-06 2024-09-13T04:58:19.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076954+0000", 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5169999999999999e-06 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076958+0000", 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1930000000000003e-06 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114300+0000", 2024-09-13T04:58:19.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037342049000000002 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114346+0000", 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5905999999999997e-05 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114389+0000", 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3482999999999997e-05 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17081 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3756032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076962+0000", 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3342631539999998, 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037593635, 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17081 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076962+0000", 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076962+0000", 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076962+0000", 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076964+0000", 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2089999999999999e-06 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076971+0000", 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2690000000000002e-06 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076976+0000", 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5990000000000004e-06 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114417+0000", 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037441601999999997 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114464+0000", 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6516e-05 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114555+0000", 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1739999999999999e-05 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17082 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3796992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076980+0000", 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3342450939999999, 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037689355000000001, 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.458 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17082 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076980+0000", 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076980+0000", 2024-09-13T04:58:19.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076980+0000", 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076982+0000", 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3379999999999999e-06 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076989+0000", 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1749999999999999e-06 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076994+0000", 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5390000000000001e-06 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114582+0000", 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037588814999999998 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114629+0000", 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6094000000000003e-05 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114669+0000", 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0726999999999998e-05 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.462 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17083 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3837952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.076998+0000", 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3342272460000002, 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037785579, 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17083 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076998+0000", 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076998+0000", 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.076997+0000", 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077000+0000", 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1730000000000002e-06 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077005+0000", 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077010+0000", 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2910000000000001e-06 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114696+0000", 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037686599000000001 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114741+0000", 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4382000000000003e-05 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114783+0000", 2024-09-13T04:58:19.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2627999999999999e-05 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17084 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3878912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.077014+0000", 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3342114550000002, 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037983981999999999, 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.467 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17084 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077014+0000", 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077014+0000", 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077013+0000", 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077015+0000", 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0600000000000002e-06 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077021+0000", 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8730000000000004e-06 2024-09-13T04:58:19.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077025+0000", 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8990000000000003e-06 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114829+0000", 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037803669999999998 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114919+0000", 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0482000000000005e-05 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.114998+0000", 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8308999999999999e-05 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17085 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3919872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.077029+0000", 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3341959189999999, 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038155294999999999, 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.472 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17085 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077029+0000", 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077029+0000", 2024-09-13T04:58:19.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077029+0000", 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077031+0000", 2024-09-13T04:58:19.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2280000000000001e-06 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077038+0000", 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4970000000000004e-06 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077043+0000", 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8069999999999999e-06 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115027+0000", 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037984163000000001 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115113+0000", 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5228000000000005e-05 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115184+0000", 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1728000000000002e-05 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17086 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 3960832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.077047+0000", 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.334177634, 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038319394999999999, 2024-09-13T04:58:19.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17086 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077047+0000", 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077047+0000", 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077047+0000", 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077055+0000", 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9629999999999998e-06 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077063+0000", 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6930000000000007e-06 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077069+0000", 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0419999999999999e-06 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115214+0000", 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038145718000000002 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115295+0000", 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0471999999999998e-05 2024-09-13T04:58:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115367+0000", 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1836999999999997e-05 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17087 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4001792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.077073+0000", 2024-09-13T04:58:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3341517409999999, 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038468527000000002, 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17087 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077073+0000", 2024-09-13T04:58:19.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077073+0000", 2024-09-13T04:58:19.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077073+0000", 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077075+0000", 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3879999999999998e-06 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077082+0000", 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0489999999999998e-06 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077087+0000", 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3989999999999997e-06 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115396+0000", 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038309334 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115472+0000", 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5870000000000004e-05 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115542+0000", 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9863999999999994e-05 2024-09-13T04:58:19.487 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17088 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4042752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.077091+0000", 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3341340480000001, 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038636503000000003, 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.488 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17088 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077091+0000", 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077091+0000", 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077091+0000", 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077093+0000", 2024-09-13T04:58:19.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3020000000000002e-06 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077101+0000", 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7100000000000007e-06 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077105+0000", 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8369999999999996e-06 2024-09-13T04:58:19.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115570+0000", 2024-09-13T04:58:19.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038464365 2024-09-13T04:58:19.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115652+0000", 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2198000000000001e-05 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115727+0000", 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5500000000000006e-05 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.493 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17089 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4083712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.077110+0000", 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3341150900000001, 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038799874999999998, 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17089 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.494 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077110+0000", 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077110+0000", 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077110+0000", 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077112+0000", 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.379e-06 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077119+0000", 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3549999999999999e-06 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077124+0000", 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7169999999999999e-06 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115758+0000", 2024-09-13T04:58:19.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038633961000000001 2024-09-13T04:58:19.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115840+0000", 2024-09-13T04:58:19.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1953000000000006e-05 2024-09-13T04:58:19.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115910+0000", 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9837000000000002e-05 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17090 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4124672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.077130+0000", 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3340953390000001, 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038957434999999999, 2024-09-13T04:58:19.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17090 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077130+0000", 2024-09-13T04:58:19.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077130+0000", 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077129+0000", 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077132+0000", 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.26e-06 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077139+0000", 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0400000000000004e-06 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077143+0000", 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6460000000000001e-06 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.115940+0000", 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038796811 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.116020+0000", 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0025999999999996e-05 2024-09-13T04:58:19.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.116087+0000", 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7012000000000001e-05 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17091 3.f 3:f13c2067:::rbd_data.112af6c12117.000000000000000b:6 [read 4165632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:19.504 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.077149+0000", 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3340760270000001, 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039120442999999998, 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17091 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.505 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077149+0000", 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077149+0000", 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077149+0000", 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077151+0000", 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3819999999999998e-06 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077158+0000", 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1110000000000002e-06 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.077163+0000", 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5059999999999997e-06 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.116113+0000", 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038950881999999999 2024-09-13T04:58:19.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.116195+0000", 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1429999999999998e-05 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.116269+0000", 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4517000000000005e-05 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.510 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.510 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:19.511 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:19.512 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:19.508+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:19.610 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-13T04:58:19.612 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:19.608+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:19.713 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:19.712+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:19.762 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28354 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738805+0000", 2024-09-13T04:58:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018232542000000001, 2024-09-13T04:58:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018260312000000001, 2024-09-13T04:58:19.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28354 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738805+0000", 2024-09-13T04:58:19.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738805+0000", 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738805+0000", 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738807+0000", 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.43e-06 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738814+0000", 2024-09-13T04:58:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4330000000000001e-06 2024-09-13T04:58:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738819+0000", 2024-09-13T04:58:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2619999999999998e-06 2024-09-13T04:58:19.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.767 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28522 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743715+0000", 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013322713999999999, 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013423384, 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28522 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743715+0000", 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743715+0000", 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743715+0000", 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743717+0000", 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8920000000000001e-06 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743722+0000", 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7559999999999999e-06 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743726+0000", 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.087e-06 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28591 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754516+0000", 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0025223279999999999, 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002684241, 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28591 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754516+0000", 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754516+0000", 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754515+0000", 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754517+0000", 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4310000000000001e-06 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754524+0000", 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.268e-06 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754529+0000", 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2870000000000004e-06 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28355 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.774 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738823+0000", 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018214442000000001, 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018437996000000002, 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28355 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738823+0000", 2024-09-13T04:58:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738823+0000", 2024-09-13T04:58:19.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738823+0000", 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738826+0000", 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5100000000000001e-06 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738832+0000", 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.404e-06 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738837+0000", 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7639999999999996e-06 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28523 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.778 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743729+0000", 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013308722, 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013595707, 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28523 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743729+0000", 2024-09-13T04:58:19.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743729+0000", 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743729+0000", 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743731+0000", 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9240000000000001e-06 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743736+0000", 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.95e-06 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743740+0000", 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9589999999999997e-06 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28592 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754534+0000", 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0025043460000000002, 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028544059999999999, 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28592 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.782 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754534+0000", 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754534+0000", 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754533+0000", 2024-09-13T04:58:19.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754536+0000", 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.694e-06 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754542+0000", 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1920000000000002e-06 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754547+0000", 2024-09-13T04:58:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2519999999999999e-06 2024-09-13T04:58:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28356 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738841+0000", 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018197076, 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018607169, 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28356 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738841+0000", 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738841+0000", 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738840+0000", 2024-09-13T04:58:19.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738843+0000", 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.261e-06 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738849+0000", 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4400000000000002e-06 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738854+0000", 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0939999999999998e-06 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28524 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743743+0000", 2024-09-13T04:58:19.789 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013294746, 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013764224, 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28524 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743743+0000", 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743743+0000", 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743743+0000", 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743749+0000", 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.241e-06 2024-09-13T04:58:19.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743755+0000", 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5119999999999999e-06 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743759+0000", 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1960000000000001e-06 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.797 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28593 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754552+0000", 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024857479999999999, 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003013005, 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28593 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754552+0000", 2024-09-13T04:58:19.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754552+0000", 2024-09-13T04:58:19.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754552+0000", 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754554+0000", 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3609999999999999e-06 2024-09-13T04:58:19.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754558+0000", 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5880000000000002e-06 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754561+0000", 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7980000000000001e-06 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28357 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738858+0000", 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018179495, 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018769330000000001, 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28357 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738858+0000", 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738858+0000", 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738858+0000", 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738860+0000", 2024-09-13T04:58:19.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3580000000000001e-06 2024-09-13T04:58:19.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738867+0000", 2024-09-13T04:58:19.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3060000000000003e-06 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738871+0000", 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7369999999999997e-06 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28525 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743781+0000", 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013257123000000001, 2024-09-13T04:58:19.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013906639, 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28525 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743781+0000", 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743781+0000", 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743780+0000", 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743783+0000", 2024-09-13T04:58:19.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9720000000000001e-06 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743790+0000", 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.251e-06 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743794+0000", 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9359999999999998e-06 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28594 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754563+0000", 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024751460000000001, 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003183647, 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28594 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754563+0000", 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754563+0000", 2024-09-13T04:58:19.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754563+0000", 2024-09-13T04:58:19.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754564+0000", 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3200000000000001e-06 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754567+0000", 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0539999999999999e-06 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754570+0000", 2024-09-13T04:58:19.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6170000000000001e-06 2024-09-13T04:58:19.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.814 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.815 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28358 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.815 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738902+0000", 2024-09-13T04:58:19.815 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018136299000000002, 2024-09-13T04:58:19.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018904234999999998, 2024-09-13T04:58:19.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28358 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738902+0000", 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738902+0000", 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738901+0000", 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738904+0000", 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4419999999999998e-06 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738913+0000", 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7250000000000003e-06 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738921+0000", 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8769999999999993e-06 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28526 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743798+0000", 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013239734, 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014066329000000001, 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28526 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743798+0000", 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743798+0000", 2024-09-13T04:58:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743798+0000", 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743800+0000", 2024-09-13T04:58:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.922e-06 2024-09-13T04:58:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743805+0000", 2024-09-13T04:58:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9570000000000001e-06 2024-09-13T04:58:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743809+0000", 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0110000000000002e-06 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28595 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754572+0000", 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024663229999999999, 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033527119999999999, 2024-09-13T04:58:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28595 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754572+0000", 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754572+0000", 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754571+0000", 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754573+0000", 2024-09-13T04:58:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.282e-06 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754576+0000", 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9170000000000002e-06 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754578+0000", 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.582e-06 2024-09-13T04:58:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28359 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738926+0000", 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018111726000000002, 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019057007000000001, 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28359 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738926+0000", 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738926+0000", 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738926+0000", 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738928+0000", 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.768e-06 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738935+0000", 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5699999999999998e-06 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738940+0000", 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1619999999999999e-06 2024-09-13T04:58:19.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28527 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743812+0000", 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013225613000000001, 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014230141999999999, 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28527 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743812+0000", 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743812+0000", 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743812+0000", 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743814+0000", 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7740000000000001e-06 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743819+0000", 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.101e-06 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743823+0000", 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1949999999999996e-06 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28596 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.836 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754580+0000", 2024-09-13T04:58:19.836 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024575249999999999, 2024-09-13T04:58:19.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035201830000000001, 2024-09-13T04:58:19.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28596 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754580+0000", 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754580+0000", 2024-09-13T04:58:19.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754580+0000", 2024-09-13T04:58:19.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:19.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754585+0000", 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7820000000000001e-06 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754588+0000", 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.529e-06 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754591+0000", 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6900000000000001e-06 2024-09-13T04:58:19.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28360 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738945+0000", 2024-09-13T04:58:19.841 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018092909000000001, 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019221367, 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28360 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738945+0000", 2024-09-13T04:58:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738945+0000", 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738945+0000", 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738947+0000", 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5859999999999999e-06 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738954+0000", 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4320000000000004e-06 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738959+0000", 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0760000000000002e-06 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28528 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743826+0000", 2024-09-13T04:58:19.846 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013211479999999999, 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014410104, 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28528 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743826+0000", 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743826+0000", 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743826+0000", 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743843+0000", 2024-09-13T04:58:19.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6665000000000001e-05 2024-09-13T04:58:19.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743850+0000", 2024-09-13T04:58:19.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1420000000000004e-06 2024-09-13T04:58:19.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743855+0000", 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6199999999999998e-06 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28597 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754593+0000", 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.002444505, 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037027869999999999, 2024-09-13T04:58:19.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28597 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754593+0000", 2024-09-13T04:58:19.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754593+0000", 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754593+0000", 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754595+0000", 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3799999999999999e-06 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754598+0000", 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0089999999999999e-06 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754600+0000", 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5100000000000001e-06 2024-09-13T04:58:19.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28361 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738965+0000", 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018072988000000002, 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019392116000000001, 2024-09-13T04:58:19.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28361 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738965+0000", 2024-09-13T04:58:19.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738965+0000", 2024-09-13T04:58:19.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738965+0000", 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738967+0000", 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5799999999999999e-06 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738974+0000", 2024-09-13T04:58:19.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4999999999999996e-06 2024-09-13T04:58:19.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738978+0000", 2024-09-13T04:58:19.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7870000000000001e-06 2024-09-13T04:58:19.861 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.861 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.861 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.862 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28529 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.862 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743859+0000", 2024-09-13T04:58:19.862 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013178898, 2024-09-13T04:58:19.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014557515, 2024-09-13T04:58:19.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28529 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743859+0000", 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743859+0000", 2024-09-13T04:58:19.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743859+0000", 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743861+0000", 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8880000000000001e-06 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743866+0000", 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.921e-06 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743870+0000", 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9090000000000002e-06 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28598 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754602+0000", 2024-09-13T04:58:19.866 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024357620000000002, 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003873133, 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28598 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754602+0000", 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754602+0000", 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754602+0000", 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754603+0000", 2024-09-13T04:58:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.55e-06 2024-09-13T04:58:19.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754606+0000", 2024-09-13T04:58:19.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0170000000000001e-06 2024-09-13T04:58:19.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754609+0000", 2024-09-13T04:58:19.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.514e-06 2024-09-13T04:58:19.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.870 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28362 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738983+0000", 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018054951999999999, 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019550643999999999, 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28362 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738983+0000", 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738983+0000", 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738983+0000", 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738985+0000", 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4870000000000001e-06 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738992+0000", 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3180000000000001e-06 2024-09-13T04:58:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738997+0000", 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28530 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743874+0000", 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013164172, 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014719179000000001, 2024-09-13T04:58:19.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28530 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743874+0000", 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743874+0000", 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743873+0000", 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743875+0000", 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.962e-06 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743880+0000", 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9089999999999999e-06 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743885+0000", 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5079999999999999e-06 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28599 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.878 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754611+0000", 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024267220000000001, 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040403000000000001, 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28599 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754611+0000", 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754611+0000", 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754611+0000", 2024-09-13T04:58:19.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754612+0000", 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2720000000000001e-06 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754615+0000", 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.971e-06 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754618+0000", 2024-09-13T04:58:19.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.441e-06 2024-09-13T04:58:19.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.882 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.882 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28363 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.883 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739002+0000", 2024-09-13T04:58:19.883 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018036304999999999, 2024-09-13T04:58:19.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019709477, 2024-09-13T04:58:19.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28363 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739002+0000", 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739002+0000", 2024-09-13T04:58:19.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739001+0000", 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739022+0000", 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0387999999999999e-05 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739030+0000", 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6319999999999997e-06 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739036+0000", 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7640000000000002e-06 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28531 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743888+0000", 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013149717999999999, 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014880908, 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28531 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743888+0000", 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743888+0000", 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743888+0000", 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743890+0000", 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9410000000000001e-06 2024-09-13T04:58:19.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743895+0000", 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8969999999999999e-06 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743899+0000", 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0509999999999998e-06 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.890 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28600 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754621+0000", 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024164770000000002, 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042068629999999999, 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28600 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754621+0000", 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754621+0000", 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754621+0000", 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754622+0000", 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.254e-06 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754626+0000", 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1329999999999999e-06 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754628+0000", 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.497e-06 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.893 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28364 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739041+0000", 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017996689, 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019844441000000001, 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28364 2024-09-13T04:58:19.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739041+0000", 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739041+0000", 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739041+0000", 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739043+0000", 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3719999999999999e-06 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739049+0000", 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2740000000000004e-06 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739054+0000", 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9119999999999997e-06 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28532 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743902+0000", 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013135449, 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015042375, 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28532 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743902+0000", 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743902+0000", 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743902+0000", 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743904+0000", 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.897e-06 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743909+0000", 2024-09-13T04:58:19.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.814e-06 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743913+0000", 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9940000000000002e-06 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28365 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739059+0000", 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017979078999999999, 2024-09-13T04:58:19.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019943836, 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28365 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739059+0000", 2024-09-13T04:58:19.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739059+0000", 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739058+0000", 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739061+0000", 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3309999999999998e-06 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739067+0000", 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9440000000000002e-06 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739071+0000", 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.651e-06 2024-09-13T04:58:19.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28533 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743916+0000", 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013121669000000001, 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015143732, 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28533 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743916+0000", 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743916+0000", 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743916+0000", 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743918+0000", 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8059999999999999e-06 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743923+0000", 2024-09-13T04:58:19.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7779999999999998e-06 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743926+0000", 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9029999999999997e-06 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28366 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739097+0000", 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017940576, 2024-09-13T04:58:19.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019993476999999999, 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28366 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739097+0000", 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739097+0000", 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739096+0000", 2024-09-13T04:58:19.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739102+0000", 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4319999999999998e-06 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739116+0000", 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4004e-05 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739125+0000", 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7350000000000002e-06 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28534 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743930+0000", 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013108303, 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015230347, 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28534 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743930+0000", 2024-09-13T04:58:19.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743930+0000", 2024-09-13T04:58:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743929+0000", 2024-09-13T04:58:19.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743931+0000", 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.849e-06 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743936+0000", 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7879999999999997e-06 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743940+0000", 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0709999999999996e-06 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28367 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739131+0000", 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017906621000000001, 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020058969999999999, 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28367 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739131+0000", 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739131+0000", 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739131+0000", 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739134+0000", 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.495e-06 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739141+0000", 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.798e-06 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739147+0000", 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2929999999999997e-06 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28535 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743944+0000", 2024-09-13T04:58:19.920 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013094267999999999, 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015278606, 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28535 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743944+0000", 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743944+0000", 2024-09-13T04:58:19.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743943+0000", 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743949+0000", 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743954+0000", 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0499999999999999e-06 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743958+0000", 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28368 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.924 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739152+0000", 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017885518999999999, 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020099395999999999, 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28368 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739152+0000", 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739152+0000", 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739152+0000", 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739155+0000", 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7599999999999998e-06 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739161+0000", 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3040000000000001e-06 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739166+0000", 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4850000000000003e-06 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28536 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743962+0000", 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013076272999999999, 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015320194000000001, 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28536 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743962+0000", 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743962+0000", 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743961+0000", 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743967+0000", 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743973+0000", 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3180000000000003e-06 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743978+0000", 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4630000000000003e-06 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28369 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739171+0000", 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017867283000000001, 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02014299, 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28369 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.932 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739171+0000", 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739171+0000", 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739170+0000", 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739173+0000", 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5720000000000001e-06 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739179+0000", 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4840000000000001e-06 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739185+0000", 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.395e-06 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28537 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743982+0000", 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013056096999999999, 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015362702000000001, 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28537 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743982+0000", 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743982+0000", 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743981+0000", 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743983+0000", 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9190000000000002e-06 2024-09-13T04:58:19.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743988+0000", 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1170000000000003e-06 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743992+0000", 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8e-06 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28370 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739190+0000", 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017848223, 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020186321, 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28370 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739190+0000", 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739190+0000", 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739189+0000", 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739192+0000", 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6280000000000001e-06 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739198+0000", 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.942e-06 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739203+0000", 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5890000000000002e-06 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28575 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.747695+0000", 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0093425350000000008, 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01170991, 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28575 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747695+0000", 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747695+0000", 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747694+0000", 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747701+0000", 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.092e-06 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747717+0000", 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6231000000000001e-05 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747727+0000", 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.251e-06 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28371 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.945 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739208+0000", 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017829919, 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020228827000000001, 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28371 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739208+0000", 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739208+0000", 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739208+0000", 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739210+0000", 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5509999999999998e-06 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739217+0000", 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9229999999999996e-06 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739222+0000", 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2789999999999998e-06 2024-09-13T04:58:19.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28576 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.747735+0000", 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0093033430000000004, 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011731614, 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28576 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747735+0000", 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747735+0000", 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747734+0000", 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747737+0000", 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2780000000000002e-06 2024-09-13T04:58:19.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747745+0000", 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4950000000000002e-06 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.747759+0000", 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.447e-05 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28372 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739227+0000", 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017810631, 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020269746000000002, 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28372 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.953 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739227+0000", 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739227+0000", 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739227+0000", 2024-09-13T04:58:19.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739229+0000", 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.65e-06 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739235+0000", 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.857e-06 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739240+0000", 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8980000000000004e-06 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28577 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754261+0000", 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0027767180000000001, 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052650270000000002, 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28577 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754261+0000", 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754261+0000", 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754260+0000", 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754265+0000", 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8760000000000004e-06 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754276+0000", 2024-09-13T04:58:19.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0453e-05 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754282+0000", 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1040000000000003e-06 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28373 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739245+0000", 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017793317999999999, 2024-09-13T04:58:19.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020312831, 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28373 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739245+0000", 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739245+0000", 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739244+0000", 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739247+0000", 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739253+0000", 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6919999999999996e-06 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739256+0000", 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0630000000000002e-06 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.962 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28578 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754285+0000", 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0027526349999999998, 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053014610000000004, 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28578 2024-09-13T04:58:19.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754285+0000", 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754285+0000", 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754285+0000", 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754287+0000", 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9769999999999999e-06 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754290+0000", 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4400000000000001e-06 2024-09-13T04:58:19.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754293+0000", 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999999e-06 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28374 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739278+0000", 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017759608999999999, 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020339258999999998, 2024-09-13T04:58:19.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28374 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739278+0000", 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739278+0000", 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739278+0000", 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739283+0000", 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1259999999999997e-06 2024-09-13T04:58:19.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739291+0000", 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6829999999999997e-06 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739298+0000", 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5320000000000003e-06 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28579 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754297+0000", 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.002740662, 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053497800000000002, 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28579 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754297+0000", 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754297+0000", 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754297+0000", 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754299+0000", 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5600000000000001e-06 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754302+0000", 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3639999999999999e-06 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754304+0000", 2024-09-13T04:58:19.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.593e-06 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28375 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.739303+0000", 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017735137000000002, 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020373571, 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28375 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739303+0000", 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739303+0000", 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739302+0000", 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739311+0000", 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.405e-06 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739318+0000", 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3440000000000003e-06 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.739323+0000", 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2129999999999999e-06 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.976 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28580 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754307+0000", 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0027311459999999998, 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054006849999999997, 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28580 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754307+0000", 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754307+0000", 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754307+0000", 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754308+0000", 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4190000000000001e-06 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754311+0000", 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9950000000000001e-06 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754314+0000", 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6699999999999998e-06 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.979 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28509 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743512+0000", 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013525446, 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016224083, 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28509 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743512+0000", 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743512+0000", 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743511+0000", 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743517+0000", 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5219999999999998e-06 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743526+0000", 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2750000000000001e-06 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743533+0000", 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0600000000000002e-06 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28581 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754317+0000", 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.002721046, 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054509119999999996, 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28581 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754317+0000", 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754317+0000", 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754317+0000", 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754319+0000", 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0959999999999999e-06 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754324+0000", 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6389999999999997e-06 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754329+0000", 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5240000000000002e-06 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28511 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743539+0000", 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013498810999999999, 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016258056999999999, 2024-09-13T04:58:19.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28511 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743539+0000", 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743539+0000", 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743539+0000", 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743541+0000", 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4049999999999998e-06 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743547+0000", 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743551+0000", 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4070000000000003e-06 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.990 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28582 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754333+0000", 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0027046399999999999, 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005494835, 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28582 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754333+0000", 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754333+0000", 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754333+0000", 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754336+0000", 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7219999999999999e-06 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754342+0000", 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3489999999999997e-06 2024-09-13T04:58:19.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754347+0000", 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7129999999999996e-06 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28513 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743555+0000", 2024-09-13T04:58:19.994 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013482839, 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016302106, 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28513 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743555+0000", 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743555+0000", 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743555+0000", 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:19.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743557+0000", 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9369999999999998e-06 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743580+0000", 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3845999999999998e-05 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743586+0000", 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0259999999999998e-06 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.997 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28583 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754350+0000", 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0026877329999999999, 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055390719999999999, 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28583 2024-09-13T04:58:19.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754350+0000", 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754350+0000", 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:19.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754350+0000", 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754352+0000", 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7910000000000001e-06 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754355+0000", 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.517e-06 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754359+0000", 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8430000000000003e-06 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28284 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 888832~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.733604+0000", 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.023434162000000001, 2024-09-13T04:58:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026319420999999999, 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28284 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.733604+0000", 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.733604+0000", 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.733603+0000", 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.733606+0000", 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2589999999999999e-06 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.733611+0000", 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6319999999999996e-06 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.733616+0000", 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.759385+0000", 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025769464999999998 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.759655+0000", 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026960100000000001 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.005 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28515 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743590+0000", 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013447549, 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016370819000000002, 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28515 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743590+0000", 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743590+0000", 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743590+0000", 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743593+0000", 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.858e-06 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743598+0000", 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3759999999999999e-06 2024-09-13T04:58:20.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743602+0000", 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1060000000000002e-06 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28584 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754368+0000", 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0026696509999999999, 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056230469999999999, 2024-09-13T04:58:20.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28584 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754368+0000", 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754368+0000", 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754368+0000", 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754371+0000", 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8839999999999998e-06 2024-09-13T04:58:20.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754378+0000", 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2280000000000001e-06 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754383+0000", 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2510000000000002e-06 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28516 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743606+0000", 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013432201, 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016417131000000001, 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28516 2024-09-13T04:58:20.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743606+0000", 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743606+0000", 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743605+0000", 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743607+0000", 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.883e-06 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743612+0000", 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0209999999999999e-06 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743616+0000", 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2050000000000004e-06 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28585 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754389+0000", 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0026491819999999999, 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056633109999999999, 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28585 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754389+0000", 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754389+0000", 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754388+0000", 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754391+0000", 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.965e-06 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754398+0000", 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.827e-06 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754403+0000", 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.186e-06 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.019 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28288 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 970752~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.734319+0000", 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.022719344999999998, 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025767498999999999, 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28288 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734319+0000", 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734319+0000", 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734317+0000", 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734324+0000", 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9319999999999999e-06 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734338+0000", 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3864e-05 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734344+0000", 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1079999999999998e-06 2024-09-13T04:58:20.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.759954+0000", 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025610162999999998 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.760037+0000", 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2767000000000002e-05 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28517 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743620+0000", 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013418019999999999, 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016501734000000001, 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28517 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.024 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743620+0000", 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743620+0000", 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743620+0000", 2024-09-13T04:58:20.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743636+0000", 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6545000000000002e-05 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743644+0000", 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9140000000000008e-06 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743649+0000", 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28586 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754408+0000", 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.002630018, 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057460660000000002, 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28586 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754408+0000", 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754408+0000", 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754408+0000", 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754410+0000", 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754417+0000", 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7309999999999996e-06 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754421+0000", 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7129999999999996e-06 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.030 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28290 3.10 3:080e56ff:::rbd_data.112af6c12117.000000000000000c:6 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.734350+0000", 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.022688327000000001, 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025840179000000001, 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28290 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734350+0000", 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734350+0000", 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734349+0000", 2024-09-13T04:58:20.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734351+0000", 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2579999999999998e-06 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734355+0000", 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0169999999999998e-06 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.734358+0000", 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9459999999999998e-06 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.760158+0000", 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025799973 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.034 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28518 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743655+0000", 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013383259999999999, 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016641316999999999, 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28518 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.035 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743655+0000", 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743655+0000", 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743654+0000", 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743657+0000", 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7080000000000002e-06 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743663+0000", 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.925e-06 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743668+0000", 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7029999999999997e-06 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28587 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754426+0000", 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0026118209999999998, 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059008300000000001, 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28587 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754426+0000", 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754426+0000", 2024-09-13T04:58:20.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754426+0000", 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754428+0000", 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5940000000000001e-06 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754435+0000", 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6839999999999999e-06 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754440+0000", 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1120000000000004e-06 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28348 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.041 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738732+0000", 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018306362, 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021624406999999998, 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28348 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738732+0000", 2024-09-13T04:58:20.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738732+0000", 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738730+0000", 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738737+0000", 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4500000000000001e-06 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738751+0000", 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4097000000000001e-05 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738760+0000", 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8980000000000002e-06 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.044 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28519 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743671+0000", 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013366546, 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016716356000000002, 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28519 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743671+0000", 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743671+0000", 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743671+0000", 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743673+0000", 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1349999999999999e-06 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743678+0000", 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3290000000000001e-06 2024-09-13T04:58:20.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743683+0000", 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0740000000000003e-06 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28588 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754459+0000", 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.002578894, 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059573009999999999, 2024-09-13T04:58:20.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28588 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754459+0000", 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754459+0000", 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754459+0000", 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754462+0000", 2024-09-13T04:58:20.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2090000000000001e-06 2024-09-13T04:58:20.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754469+0000", 2024-09-13T04:58:20.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7239999999999992e-06 2024-09-13T04:58:20.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754475+0000", 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3739999999999997e-06 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28350 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738767+0000", 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018270859, 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021680464999999999, 2024-09-13T04:58:20.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28350 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738767+0000", 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738767+0000", 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738766+0000", 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738770+0000", 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1700000000000001e-06 2024-09-13T04:58:20.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738777+0000", 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9530000000000002e-06 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738782+0000", 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3589999999999999e-06 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28520 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743686+0000", 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013351725, 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016790382, 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28520 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743686+0000", 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743686+0000", 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743686+0000", 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743688+0000", 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9850000000000001e-06 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743693+0000", 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0730000000000004e-06 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743697+0000", 2024-09-13T04:58:20.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8319999999999999e-06 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28589 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754480+0000", 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0025582410000000002, 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060277880000000001, 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.059 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28589 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754480+0000", 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754480+0000", 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754479+0000", 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754482+0000", 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7779999999999999e-06 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754489+0000", 2024-09-13T04:58:20.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.81e-06 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754494+0000", 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9989999999999999e-06 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28352 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.738787+0000", 2024-09-13T04:58:20.062 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018251191, 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021749775999999998, 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28352 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738787+0000", 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738787+0000", 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738786+0000", 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738789+0000", 2024-09-13T04:58:20.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.655e-06 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738795+0000", 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5239999999999997e-06 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.738800+0000", 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28521 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.743701+0000", 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.013337366999999999, 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016866711, 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28521 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743701+0000", 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743701+0000", 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743700+0000", 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743702+0000", 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8500000000000001e-06 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743707+0000", 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.062e-06 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.743712+0000", 2024-09-13T04:58:20.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.408e-06 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28590 3.10 3.ff6a7010 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:19.754498+0000", 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0025397250000000001, 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006097898, 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28590 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754498+0000", 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754498+0000", 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754498+0000", 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754500+0000", 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.249e-06 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754506+0000", 2024-09-13T04:58:20.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8490000000000002e-06 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:19.754511+0000", 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0549999999999999e-06 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 80 2024-09-13T04:58:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:20.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:800 trunc from 0x6985e74 to 0xd74ba8d 2024-09-13T04:58:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:801 trunc from 0xd74ba8d to 0xe289419 2024-09-13T04:58:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:802 read 0x73a1216 thru 0x73aa1c8 (0x8fb3 bytes) 2024-09-13T04:58:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:803 write 0x7c30196 thru 0x7c3c38d (0xc1f8 bytes) 2024-09-13T04:58:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:805 write 0x8c56a82 thru 0x8c5a68d (0x3c0c bytes) 2024-09-13T04:58:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:806 writesame 0x6406a2a thru 0x6409406 data_size 0x1(0x29dd bytes) 2024-09-13T04:58:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:807 write 0xcf473f thru 0xd00bf4 (0xc4b6 bytes) 2024-09-13T04:58:20.076 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:20.072+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:20.113 INFO:teuthology.orchestra.run.smithi001.stdout:808 clone 6 order 20 su 524288 sc 15 2024-09-13T04:58:20.174 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:20.172+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:20.244 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:20.244 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:20.244 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:20.244 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:20.244 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:20.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:20.275 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:20.276+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:20.326 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:58:20.397 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:20.396+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:20.439 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone5 from 0xe289419 (overlap 0x6985e74) to 0x546b1f6 2024-09-13T04:58:20.444 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:20.444 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:20.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:20.444 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:20.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16592 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4067780169999997, 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055701260000000002, 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16592 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036513+0000", 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5299999999999999e-06 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036521+0000", 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6699999999999994e-06 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036525+0000", 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3610000000000003e-06 2024-09-13T04:58:20.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092051+0000", 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05552613 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092163+0000", 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111494 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092212+0000", 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9432999999999999e-05 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16593 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4067594970000004, 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055799934000000002, 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16593 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036532+0000", 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3810000000000002e-06 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036539+0000", 2024-09-13T04:58:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2139999999999999e-06 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036544+0000", 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1909999999999999e-06 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092237+0000", 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692609999999997 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092290+0000", 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3554000000000003e-05 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092329+0000", 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9335999999999997e-05 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16594 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4067405229999999, 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055959726000000001, 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16594 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036550+0000", 2024-09-13T04:58:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036557+0000", 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999999e-06 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036562+0000", 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092364+0000", 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801515000000003 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092441+0000", 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7044000000000003e-05 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092508+0000", 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7617e-05 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16595 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4063888149999997, 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055757809999999998, 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16595 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036566+0000", 2024-09-13T04:58:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996657 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036905+0000", 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033883999999999997 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036915+0000", 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0164e-05 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036930+0000", 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5245999999999999e-05 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092559+0000", 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055628628999999999 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092615+0000", 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6684000000000001e-05 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092658+0000", 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2559999999999999e-05 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16596 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3215360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4061511700000002, 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696469999999998, 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16596 2024-09-13T04:58:20.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.463 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036936+0000", 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997983 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037142+0000", 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020614900000000001 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037152+0000", 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6739999999999992e-06 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037159+0000", 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092692+0000", 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055533569999999997 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092768+0000", 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5220000000000001e-05 2024-09-13T04:58:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092834+0000", 2024-09-13T04:58:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6674999999999994e-05 2024-09-13T04:58:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16597 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4059983090000001, 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055678257000000002, 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16597 2024-09-13T04:58:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037164+0000", 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998736 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037300+0000", 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000135456 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037310+0000", 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0366e-05 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037316+0000", 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4910000000000002e-06 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092888+0000", 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055571872000000001 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092929+0000", 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0667999999999998e-05 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092969+0000", 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9913000000000002e-05 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16603 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4049699350000004, 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055630449999999998, 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16603 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038321+0000", 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1179999999999999e-06 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038326+0000", 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6189999999999999e-06 2024-09-13T04:58:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038331+0000", 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2429999999999999e-06 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093860+0000", 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055528871 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093900+0000", 2024-09-13T04:58:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0742999999999999e-05 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093949+0000", 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16604 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4049543059999996, 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055794816999999997, 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16604 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038334+0000", 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038336+0000", 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.046e-06 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038342+0000", 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038346+0000", 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1799999999999998e-06 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093983+0000", 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055636906999999999 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094059+0000", 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6075999999999995e-05 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094129+0000", 2024-09-13T04:58:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0337000000000001e-05 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16606 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.404445494, 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055591046999999998, 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16606 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038845+0000", 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4329999999999998e-06 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038852+0000", 2024-09-13T04:58:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1240000000000002e-06 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038857+0000", 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9620000000000001e-06 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094307+0000", 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055450714999999998 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094392+0000", 2024-09-13T04:58:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4586000000000006e-05 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094435+0000", 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2697000000000001e-05 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16607 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4044283560000004, 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696915999999999, 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16607 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038860+0000", 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038862+0000", 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1859999999999999e-06 2024-09-13T04:58:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038868+0000", 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8590000000000001e-06 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038873+0000", 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6310000000000002e-06 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094459+0000", 2024-09-13T04:58:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055585561999999998 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094519+0000", 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0161999999999998e-05 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094558+0000", 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8831999999999997e-05 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16608 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4044120969999998, 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801370000000003, 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16608 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038879+0000", 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.21e-06 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038885+0000", 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8100000000000003e-06 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038889+0000", 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2760000000000002e-06 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094581+0000", 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692236999999999 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094639+0000", 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7935e-05 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094678+0000", 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9267999999999997e-05 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16609 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4041822210000001, 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055693234000000001, 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16609 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038892+0000", 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997854 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039112+0000", 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000219402 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039125+0000", 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039135+0000", 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8549999999999997e-06 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094702+0000", 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055567083000000003 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094761+0000", 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9425000000000001e-05 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094800+0000", 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8754999999999997e-05 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16610 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4040971139999998, 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0557396, 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16610 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039158+0000", 2024-09-13T04:58:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999666 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039196+0000", 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7551e-05 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039212+0000", 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6569999999999999e-05 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039219+0000", 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5080000000000002e-06 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094824+0000", 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055604690999999998 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094890+0000", 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6204999999999994e-05 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094931+0000", 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1675000000000003e-05 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16636 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4003883049999999, 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055573821000000002, 2024-09-13T04:58:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16636 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042900+0000", 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042902+0000", 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0590000000000001e-06 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042908+0000", 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5620000000000003e-06 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042912+0000", 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.437e-06 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098321+0000", 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055408959000000001 2024-09-13T04:58:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098401+0000", 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0122000000000003e-05 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098475+0000", 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3002000000000003e-05 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16638 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.400056856, 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055689097, 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16638 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043234+0000", 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043237+0000", 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6260000000000001e-06 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043240+0000", 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.627e-06 2024-09-13T04:58:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098773+0000", 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055532519000000002 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098849+0000", 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6808000000000001e-05 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098921+0000", 2024-09-13T04:58:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1835000000000006e-05 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16683 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.3920059599999997, 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055570913999999999, 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16683 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051285+0000", 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051294+0000", 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2239999999999995e-06 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051299+0000", 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0259999999999998e-06 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106697+0000", 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055398672000000003 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106781+0000", 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3849000000000003e-05 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106854+0000", 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2683999999999997e-05 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16684 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.3919854970000003, 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055767156999999998, 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16684 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051305+0000", 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.204e-06 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051313+0000", 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4830000000000002e-06 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051318+0000", 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6529999999999999e-06 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106896+0000", 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055577731999999998 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106987+0000", 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0920000000000004e-05 2024-09-13T04:58:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107071+0000", 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3492999999999995e-05 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16685 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.3919663480000004, 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055971087000000003, 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16685 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051322+0000", 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051325+0000", 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4499999999999998e-06 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051332+0000", 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2409999999999998e-06 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051337+0000", 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0499999999999999e-06 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107129+0000", 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055791855000000001 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107212+0000", 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3046000000000004e-05 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107294+0000", 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1773000000000005e-05 2024-09-13T04:58:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16689 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.3907020839999999, 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694117000000001, 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16689 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052589+0000", 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4420000000000001e-06 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052595+0000", 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8549999999999998e-06 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052600+0000", 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8169999999999998e-06 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108111+0000", 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055511660999999997 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108201+0000", 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9386000000000001e-05 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108281+0000", 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0338000000000002e-05 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16691 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.3902858409999999, 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055651254999999997, 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16691 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053005+0000", 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4490000000000002e-06 2024-09-13T04:58:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053010+0000", 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2680000000000002e-06 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053014+0000", 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9729999999999999e-06 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108498+0000", 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055483769000000002 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108580+0000", 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1638999999999995e-05 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108654+0000", 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4548000000000005e-05 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:20.538 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:20.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:20.540 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:20.536+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:20.640 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:20.636+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:20.705 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:20.705 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:20.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:20.765 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:20.764+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:20.791 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-13T04:58:20.811 INFO:teuthology.orchestra.run.smithi001.stdout:809 read 0x1d71136 thru 0x1d728fd (0x17c8 bytes) 2024-09-13T04:58:20.816 INFO:teuthology.orchestra.run.smithi001.stdout:810 trunc from 0xe289419 to 0xc8a1e56 2024-09-13T04:58:20.842 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:20.840+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:20.848 INFO:teuthology.orchestra.run.smithi001.stdout:811 read 0xa890846 thru 0xa8980a8 (0x7863 bytes) 2024-09-13T04:58:20.865 INFO:teuthology.orchestra.run.smithi001.stdout:812 trunc from 0xc8a1e56 to 0xbb4768 2024-09-13T04:58:20.911 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:20.912 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:20.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:20.943 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:20.940+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:20.971 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:20.997 INFO:teuthology.orchestra.run.smithi001.stdout:814 write 0xcec98f9 thru 0xced18be (0x7fc6 bytes) 2024-09-13T04:58:21.001 INFO:teuthology.orchestra.run.smithi001.stdout:815 read 0x448e15f thru 0x4490d08 (0x2baa bytes) 2024-09-13T04:58:21.001 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.002 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.002 INFO:teuthology.orchestra.run.smithi001.stdout:819 write 0x85a8a64 thru 0x85b1e0c (0x93a9 bytes) 2024-09-13T04:58:21.011 INFO:teuthology.orchestra.run.smithi001.stdout:820 write 0x408b434 thru 0x4095202 (0x9dcf bytes) 2024-09-13T04:58:21.021 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.021 INFO:teuthology.orchestra.run.smithi001.stdout:822 write 0xaefcc5f thru 0xaf02c83 (0x6025 bytes) 2024-09-13T04:58:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:823 read 0x225e4b7 thru 0x2268643 (0xa18d bytes) 2024-09-13T04:58:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.030 INFO:teuthology.orchestra.run.smithi001.stdout:826 trunc from 0xced18bf to 0xb4945bd 2024-09-13T04:58:21.044 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:21.040+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:21.058 INFO:teuthology.orchestra.run.smithi001.stdout:827 writesame 0xd6c62b3 thru 0xd6cd49e data_size 0x2e(0x71ec bytes) 2024-09-13T04:58:21.058 INFO:teuthology.orchestra.run.smithi001.stdout:828 read 0x26f043e thru 0x26fbb13 (0xb6d6 bytes) 2024-09-13T04:58:21.058 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.058 INFO:teuthology.orchestra.run.smithi001.stdout:831 punch from 0xae5519c to 0xae5d15b, (0x7fbf bytes) 2024-09-13T04:58:21.058 INFO:teuthology.orchestra.run.smithi001.stdout:832 write 0x6aba174 thru 0x6aba1cf (0x5c bytes) 2024-09-13T04:58:21.063 INFO:teuthology.orchestra.run.smithi001.stdout:833 write 0xb7ea3f0 thru 0xb7f798a (0xd59b bytes) 2024-09-13T04:58:21.074 INFO:teuthology.orchestra.run.smithi001.stdout:834 writesame 0xc555edb thru 0xc55b05a data_size 0x20(0x5180 bytes) 2024-09-13T04:58:21.077 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:21.082 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.082 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.082 INFO:teuthology.orchestra.run.smithi001.stdout:839 write 0x54eb866 thru 0x54f4dbc (0x9557 bytes) 2024-09-13T04:58:21.089 INFO:teuthology.orchestra.run.smithi001.stdout:840 write 0xb2e7efe thru 0xb2eb442 (0x3545 bytes) 2024-09-13T04:58:21.105 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:1134 3.16 3:6b6d2fa2:::rbd_data.112ac9124348.000000000000000c:head [set-alloc-hint object_size 8388608 write_size 8388608,write 4917978~55166] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:10.801349+0000", 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.301106251, 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051958339999999999, 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1134 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801349+0000", 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801349+0000", 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801347+0000", 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801396+0000", 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8782000000000002e-05 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801415+0000", 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8768000000000001e-05 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801424+0000", 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3039999999999994e-06 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801507+0000", 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2582000000000003e-05 2024-09-13T04:58:21.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801649+0000", 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000142142 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.801912+0000", 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026283900000000001 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.806396+0000", 2024-09-13T04:58:21.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044844220000000001 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.806409+0000", 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2602999999999999e-05 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.806485+0000", 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6274999999999997e-05 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:10.806545+0000", 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9571000000000003e-05 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:13769 3.5 3:a55a4cd2:::rbd_data.112a2c87bb71.0000000000000120:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.3586487600000003, 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007083972, 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13769 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:21.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743805+0000", 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743810+0000", 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0540000000000002e-06 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743823+0000", 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.275e-05 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743831+0000", 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6750000000000002e-06 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743877+0000", 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6597000000000001e-05 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743956+0000", 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8732000000000004e-05 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.744121+0000", 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016536900000000001 2024-09-13T04:58:21.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.745356+0000", 2024-09-13T04:58:21.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001234824 2024-09-13T04:58:21.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750813+0000", 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00545671 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750854+0000", 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1309e-05 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750890+0000", 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5927999999999999e-05 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.6412959560000004, 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:58:21.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:21.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:58:21.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:21.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:58:21.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:58:21.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:58:21.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:58:21.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:58:21.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28635 3.16 3:6d9fd88d:::rbd_object_map.112a547f4795:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.013718+0000", 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.0887373250000001, 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055726129999999997, 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.126 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28635 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.013718+0000", 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.013718+0000", 2024-09-13T04:58:21.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.013716+0000", 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.013726+0000", 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0032e-05 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.013749+0000", 2024-09-13T04:58:21.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2558999999999999e-05 2024-09-13T04:58:21.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.013765+0000", 2024-09-13T04:58:21.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5766000000000001e-05 2024-09-13T04:58:21.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.016326+0000", 2024-09-13T04:58:21.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002560933 2024-09-13T04:58:21.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.016472+0000", 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146274 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.017292+0000", 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00081968199999999996 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.018844+0000", 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001552606 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.019164+0000", 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000320212 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.019237+0000", 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2664999999999996e-05 2024-09-13T04:58:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.019290+0000", 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3284e-05 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28802 3.16 3:6d9fd88d:::rbd_object_map.112a547f4795:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.087971+0000", 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.01448429, 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060163980000000001, 2024-09-13T04:58:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28802 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.087971+0000", 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.087971+0000", 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.087968+0000", 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.087985+0000", 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.696e-05 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088018+0000", 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3485000000000002e-05 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088042+0000", 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3618999999999999e-05 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088185+0000", 2024-09-13T04:58:21.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014307 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088450+0000", 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026445400000000001 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.089540+0000", 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010899270000000001 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.093626+0000", 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040866440000000004 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.093739+0000", 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112417 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.093883+0000", 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000144714 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.093987+0000", 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103805 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28803 3.16 3:6d9fd88d:::rbd_object_map.112a547f4795:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.088577+0000", 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.013877648, 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066222870000000001, 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28803 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088577+0000", 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088577+0000", 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088575+0000", 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088589+0000", 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4249e-05 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088617+0000", 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7121999999999999e-05 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088630+0000", 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.348e-05 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.090888+0000", 2024-09-13T04:58:21.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022581559999999999 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.091135+0000", 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024649900000000001 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.092384+0000", 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012491729999999999 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.094550+0000", 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021664639999999999 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.094997+0000", 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044710300000000002 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.095137+0000", 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013970000000000001 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.095200+0000", 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2557000000000004e-05 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30534 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823146+0000", 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27930935499999998, 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083493909999999994, 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30534 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823146+0000", 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823146+0000", 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823144+0000", 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823153+0000", 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7660000000000004e-06 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823172+0000", 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8374999999999999e-05 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823182+0000", 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0732e-05 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823237+0000", 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4301000000000001e-05 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823324+0000", 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7865e-05 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823781+0000", 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045668299999999999 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827196+0000", 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034147930000000002 2024-09-13T04:58:21.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831407+0000", 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042107070000000002 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831456+0000", 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9398999999999999e-05 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831495+0000", 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9016e-05 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30535 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823219+0000", 2024-09-13T04:58:21.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27923615000000002, 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010033459999999999, 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30535 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823219+0000", 2024-09-13T04:58:21.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823219+0000", 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823218+0000", 2024-09-13T04:58:21.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823225+0000", 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0809999999999996e-06 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823238+0000", 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2788e-05 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823247+0000", 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824276+0000", 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010286920000000001 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824357+0000", 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1415999999999994e-05 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824623+0000", 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026604299999999999 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827205+0000", 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002581421 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833190+0000", 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059849150000000004 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833219+0000", 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9339000000000001e-05 2024-09-13T04:58:21.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833252+0000", 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3249e-05 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.158 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30536 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823272+0000", 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.279183511, 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010117332999999999, 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30536 2024-09-13T04:58:21.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823272+0000", 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823272+0000", 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823271+0000", 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823276+0000", 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823288+0000", 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2014999999999999e-05 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823297+0000", 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4700000000000002e-06 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824906+0000", 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001609463 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824974+0000", 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8053999999999996e-05 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.825237+0000", 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026292399999999999 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828888+0000", 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036508140000000001 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833301+0000", 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004413152 2024-09-13T04:58:21.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833348+0000", 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7144999999999997e-05 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833389+0000", 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0586000000000001e-05 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30539 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823354+0000", 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27910144999999997, 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012316781000000001, 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30539 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.166 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823354+0000", 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823354+0000", 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823352+0000", 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823359+0000", 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.63e-06 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823372+0000", 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2948e-05 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823381+0000", 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5140000000000001e-06 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.825528+0000", 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002147693 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.825575+0000", 2024-09-13T04:58:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6873999999999999e-05 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.825839+0000", 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026439299999999997 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828899+0000", 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030599809999999998 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835606+0000", 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067060399999999999 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835634+0000", 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8365999999999999e-05 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835670+0000", 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6464999999999997e-05 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.171 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30540 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823479+0000", 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27897554699999999, 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012343474, 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30540 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823479+0000", 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823479+0000", 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823478+0000", 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823487+0000", 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4030000000000007e-06 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823517+0000", 2024-09-13T04:58:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9878000000000001e-05 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823527+0000", 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0536999999999999e-05 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.826173+0000", 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026460680000000001 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.826302+0000", 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012925 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.826743+0000", 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044028999999999997 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830129+0000", 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003385805 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835719+0000", 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055906890000000002 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835768+0000", 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9162999999999997e-05 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835823+0000", 2024-09-13T04:58:21.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4552999999999998e-05 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30541 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823556+0000", 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27889881100000002, 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012393184, 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30541 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823556+0000", 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823556+0000", 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823555+0000", 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823562+0000", 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9480000000000002e-06 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823574+0000", 2024-09-13T04:58:21.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2047e-05 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823582+0000", 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9249999999999995e-06 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827251+0000", 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036688440000000001 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827342+0000", 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0674000000000006e-05 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827634+0000", 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000292358 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830131+0000", 2024-09-13T04:58:21.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024971989999999999 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835874+0000", 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005742601 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835912+0000", 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8052000000000001e-05 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835949+0000", 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7370000000000003e-05 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30542 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823605+0000", 2024-09-13T04:58:21.184 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27884962600000002, 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012485001000000001, 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30542 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823605+0000", 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823605+0000", 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823605+0000", 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823611+0000", 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4060000000000002e-06 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823622+0000", 2024-09-13T04:58:21.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1221e-05 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823630+0000", 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8110000000000003e-06 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827943+0000", 2024-09-13T04:58:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043126789999999998 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828041+0000", 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8475999999999997e-05 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828478+0000", 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00043719599999999999 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831354+0000", 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028756250000000001 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836007+0000", 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046533980000000004 2024-09-13T04:58:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836055+0000", 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8007000000000003e-05 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836090+0000", 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4929999999999999e-05 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.191 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30543 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823638+0000", 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27881750300000002, 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01329027, 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30543 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823638+0000", 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823638+0000", 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823637+0000", 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823642+0000", 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7709999999999997e-06 2024-09-13T04:58:21.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823655+0000", 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3414000000000001e-05 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823663+0000", 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5179999999999998e-06 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828946+0000", 2024-09-13T04:58:21.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052829060000000004 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829034+0000", 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7979000000000006e-05 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829300+0000", 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026641600000000001 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831357+0000", 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020569590000000001 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836829+0000", 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054724259999999999 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836893+0000", 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3313999999999997e-05 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836928+0000", 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5157000000000002e-05 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.198 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30544 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823686+0000", 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27876864400000001, 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013377933, 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30544 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823686+0000", 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823686+0000", 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823685+0000", 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823691+0000", 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0009999999999998e-06 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823702+0000", 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0672999999999999e-05 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823710+0000", 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5939999999999996e-06 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829632+0000", 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059225720000000001 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829683+0000", 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1242000000000002e-05 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829886+0000", 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020243799999999999 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.832049+0000", 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021627619999999999 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836980+0000", 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004930963 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.837028+0000", 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8659999999999998e-05 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.837064+0000", 2024-09-13T04:58:21.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6127999999999997e-05 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30545 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823717+0000", 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27873810999999998, 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01503137, 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.205 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30545 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823717+0000", 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823717+0000", 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823716+0000", 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823721+0000", 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823730+0000", 2024-09-13T04:58:21.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.037e-06 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823754+0000", 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3283000000000001e-05 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830192+0000", 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064381430000000003 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830290+0000", 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8552000000000001e-05 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830560+0000", 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026991899999999997 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.832051+0000", 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001490901 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838665+0000", 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006613687 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838711+0000", 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6536000000000003e-05 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838748+0000", 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6788000000000001e-05 2024-09-13T04:58:21.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30546 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823776+0000", 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27867919699999999, 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015108465, 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30546 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823776+0000", 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823776+0000", 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823775+0000", 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823780+0000", 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1039999999999998e-06 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823807+0000", 2024-09-13T04:58:21.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6262e-05 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823817+0000", 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8020000000000003e-06 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830869+0000", 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070525789999999998 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830922+0000", 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3183000000000002e-05 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831111+0000", 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018851100000000001 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.832056+0000", 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094489200000000002 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838811+0000", 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067551410000000001 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838852+0000", 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0874000000000002e-05 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838884+0000", 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2598999999999997e-05 2024-09-13T04:58:21.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30547 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823825+0000", 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.27863040100000003, 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01570467, 2024-09-13T04:58:21.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30547 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823825+0000", 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823825+0000", 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823824+0000", 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823830+0000", 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.443e-06 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823840+0000", 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0132000000000001e-05 2024-09-13T04:58:21.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823846+0000", 2024-09-13T04:58:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6919999999999999e-06 2024-09-13T04:58:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831534+0000", 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0076881500000000004 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831593+0000", 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9419000000000001e-05 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831828+0000", 2024-09-13T04:58:21.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000234229 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.832476+0000", 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00064819699999999999 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839440+0000", 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069641629999999998 2024-09-13T04:58:21.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839490+0000", 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9685000000000003e-05 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839529+0000", 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9578e-05 2024-09-13T04:58:21.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30552 3.5 3:a4e32c5e:::rbd_data.112a1dc60142.00000000000000c5:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.833493+0000", 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.26896173200000001, 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064428080000000004, 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30552 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833493+0000", 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833493+0000", 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833492+0000", 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833498+0000", 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8719999999999999e-06 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833509+0000", 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0664e-05 2024-09-13T04:58:21.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833514+0000", 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2349999999999999e-06 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833547+0000", 2024-09-13T04:58:21.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3285999999999999e-05 2024-09-13T04:58:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833640+0000", 2024-09-13T04:58:21.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2330000000000005e-05 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833750+0000", 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011045899999999999 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.834144+0000", 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039349600000000002 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839822+0000", 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005678336 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839900+0000", 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8448000000000005e-05 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839936+0000", 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5697000000000001e-05 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.233 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30978 3.5 3:a140ae6c:::rbd_data.112a1dc60142.0000000000000084:head [set-alloc-hint object_size 1048576 write_size 1048576,write 690788~37801] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:21.006253+0000", 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.096201718000000006, 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062508579999999998, 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30978 2024-09-13T04:58:21.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006253+0000", 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006253+0000", 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:21.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006252+0000", 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006271+0000", 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8952000000000001e-05 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006286+0000", 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4960000000000001e-05 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006294+0000", 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9999999999999996e-06 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006327+0000", 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3285999999999999e-05 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006405+0000", 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7645000000000006e-05 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006484+0000", 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8464e-05 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.008310+0000", 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018266860000000001 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.012456+0000", 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041456770000000004 2024-09-13T04:58:21.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.012484+0000", 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8204e-05 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.012504+0000", 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0075e-05 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:21.239 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:21.240 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:21.240 INFO:teuthology.orchestra.run.smithi001.stdout:841 trunc from 0xd6cd49f to 0x48178ce 2024-09-13T04:58:21.241 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:21.242 INFO:teuthology.orchestra.run.smithi001.stdout:843 trunc from 0x48178ce to 0xea49acc 2024-09-13T04:58:21.242 INFO:teuthology.orchestra.run.smithi001.stdout:844 flatten 2024-09-13T04:58:21.242 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:21.243 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:21.243 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:21.240+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:21.256 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:21.257 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:21.275 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:21.293 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:21.314 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:58:21.317 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:21.317 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:21.341 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:21.341 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:21.341 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:21.340+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:21.355 INFO:teuthology.orchestra.run.smithi001.stdout:846 write 0x8d6ac8b thru 0x8d6dfc6 (0x333c bytes) 2024-09-13T04:58:21.362 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:21.362 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:21.365 INFO:teuthology.orchestra.run.smithi001.stdout:848 trunc from 0xea49acc to 0xb652d04 2024-09-13T04:58:21.373 INFO:teuthology.orchestra.run.smithi001.stdout:849 punch from 0xb0e208b to 0xb0e8788, (0x66fd bytes) 2024-09-13T04:58:21.373 INFO:teuthology.orchestra.run.smithi001.stdout:850 punch from 0x384419a to 0x384a591, (0x63f7 bytes) 2024-09-13T04:58:21.373 INFO:teuthology.orchestra.run.smithi001.stdout:852 read 0xfeb06 thru 0x102735 (0x3c30 bytes) 2024-09-13T04:58:21.374 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.374 INFO:teuthology.orchestra.run.smithi001.stdout:854 write 0xd39686 thru 0xd40a07 (0x7382 bytes) 2024-09-13T04:58:21.386 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:21.386 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:21.387 INFO:teuthology.orchestra.run.smithi001.stdout:855 write 0xde162fa thru 0xde20366 (0xa06d bytes) 2024-09-13T04:58:21.391 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:21.391 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:21.394 INFO:teuthology.orchestra.run.smithi001.stdout:856 write 0x4382c9b thru 0x438d285 (0xa5eb bytes) 2024-09-13T04:58:21.400 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.400 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.401 INFO:teuthology.orchestra.run.smithi001.stdout:859 write 0x2db552d thru 0x2dbc969 (0x743d bytes) 2024-09-13T04:58:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:860 trunc from 0xde20367 to 0xd2f62d4 2024-09-13T04:58:21.411 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:21.420 INFO:teuthology.orchestra.run.smithi001.stdout:861 read 0x5deeca thru 0x5e1935 (0x2a6c bytes) 2024-09-13T04:58:21.429 INFO:teuthology.orchestra.run.smithi001.stdout:863 write 0xe56e830 thru 0xe57e25c (0xfa2d bytes) 2024-09-13T04:58:21.435 INFO:teuthology.orchestra.run.smithi001.stdout:864 writesame 0xbb3d446 thru 0xbb488cb data_size 0x7(0xb486 bytes) 2024-09-13T04:58:21.442 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:21.440+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:21.446 INFO:teuthology.orchestra.run.smithi001.stdout:865 write 0x1710766 thru 0x1720638 (0xfed3 bytes) 2024-09-13T04:58:21.459 INFO:teuthology.orchestra.run.smithi001.stdout:869 writesame 0x303695d thru 0x304456f data_size 0x1(0xdc13 bytes) 2024-09-13T04:58:21.498 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.498 INFO:teuthology.orchestra.run.smithi001.stdout:871 punch from 0x5291846 to 0x5291ae6, (0x2a0 bytes) 2024-09-13T04:58:21.498 INFO:teuthology.orchestra.run.smithi001.stdout:872 read 0xdc29e9d thru 0xdc38fa0 (0xf104 bytes) 2024-09-13T04:58:21.498 INFO:teuthology.orchestra.run.smithi001.stdout:873 read 0x309ca4c thru 0x30a0aae (0x4063 bytes) 2024-09-13T04:58:21.498 INFO:teuthology.orchestra.run.smithi001.stdout:874 punch from 0xa0dfc11 to 0xa0e1431, (0x1820 bytes) 2024-09-13T04:58:21.498 INFO:teuthology.orchestra.run.smithi001.stdout:875 writesame 0x7c86a9d thru 0x7c88746 data_size 0x3(0x1caa bytes) 2024-09-13T04:58:21.508 INFO:teuthology.orchestra.run.smithi001.stdout:876 write 0xbb22f7e thru 0xbb2ca94 (0x9b17 bytes) 2024-09-13T04:58:21.516 INFO:teuthology.orchestra.run.smithi001.stdout:877 write 0x5d3883f thru 0x5d38ff7 (0x7b9 bytes) 2024-09-13T04:58:21.532 INFO:teuthology.orchestra.run.smithi001.stdout:878 trunc from 0xe57e25d to 0x9b6a104 2024-09-13T04:58:21.544 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:21.544+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:21.552 INFO:teuthology.orchestra.run.smithi001.stdout:879 read 0x484d4f7 thru 0x4852ba2 (0x56ac bytes) 2024-09-13T04:58:21.554 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:21.554 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:21.554 INFO:teuthology.orchestra.run.smithi001.stdout:880 trunc from 0x9b6a104 to 0x9433376 2024-09-13T04:58:21.571 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:21.576 INFO:teuthology.orchestra.run.smithi001.stdout:881 trunc from 0x9433376 to 0x4692f49 2024-09-13T04:58:21.595 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.595 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.595 INFO:teuthology.orchestra.run.smithi001.stdout:885 read 0x3144c5c thru 0x3151a90 (0xce35 bytes) 2024-09-13T04:58:21.601 INFO:teuthology.orchestra.run.smithi001.stdout:888 trunc from 0x4692f49 to 0x6ec873 2024-09-13T04:58:21.645 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:21.644+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:21.646 INFO:teuthology.orchestra.run.smithi001.stdout:889 write 0x83f4ac8 thru 0x83f7b84 (0x30bd bytes) 2024-09-13T04:58:21.661 INFO:teuthology.orchestra.run.smithi001.stdout:890 trunc from 0x83f7b85 to 0xd911022 2024-09-13T04:58:21.669 INFO:teuthology.orchestra.run.smithi001.stdout:893 writesame 0x14ca4b5 thru 0x14cf29c data_size 0x18(0x4de8 bytes) 2024-09-13T04:58:21.677 INFO:teuthology.orchestra.run.smithi001.stdout:894 write 0x2b120b8 thru 0x2b21641 (0xf58a bytes) 2024-09-13T04:58:21.688 INFO:teuthology.orchestra.run.smithi001.stdout:895 write 0x46ce13f thru 0x46dd693 (0xf555 bytes) 2024-09-13T04:58:21.701 INFO:teuthology.orchestra.run.smithi001.stdout:896 read 0xbd97833 thru 0xbd9b7c5 (0x3f93 bytes) 2024-09-13T04:58:21.707 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:21.708 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:21.720 INFO:teuthology.orchestra.run.smithi001.stdout:898 trunc from 0xd911022 to 0x3cb0f5d 2024-09-13T04:58:21.729 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:21.749 INFO:teuthology.orchestra.run.smithi001.stdout:899 read 0x18fb67 thru 0x1970dc (0x7576 bytes) 2024-09-13T04:58:21.749 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:21.748+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:21.750 INFO:teuthology.orchestra.run.smithi001.stdout:901 read 0x3f75de thru 0x403e78 (0xc89b bytes) 2024-09-13T04:58:21.751 INFO:teuthology.orchestra.run.smithi001.stdout:902 read 0xd0615 thru 0xd8d28 (0x8714 bytes) 2024-09-13T04:58:21.751 INFO:teuthology.orchestra.run.smithi001.stdout:903 read 0x271f75e thru 0x272decb (0xe76e bytes) 2024-09-13T04:58:21.751 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.752 INFO:teuthology.orchestra.run.smithi001.stdout:905 punch from 0xc24302 to 0xc2e2e9, (0x9fe7 bytes) 2024-09-13T04:58:21.752 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.752 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.772 INFO:teuthology.orchestra.run.smithi001.stdout:908 write 0x9a0f5a3 thru 0x9a14d61 (0x57bf bytes) 2024-09-13T04:58:21.778 INFO:teuthology.orchestra.run.smithi001.stdout:909 write 0x5b57bad thru 0x5b63c6b (0xc0bf bytes) 2024-09-13T04:58:21.806 INFO:teuthology.orchestra.run.smithi001.stdout:910 trunc from 0x9a14d62 to 0x1a0bc24 2024-09-13T04:58:21.832 INFO:teuthology.orchestra.run.smithi001.stdout:911 read 0x15f01b7 thru 0x15fe4b6 (0xe300 bytes) 2024-09-13T04:58:21.832 INFO:teuthology.orchestra.run.smithi001.stdout:912 punch from 0x15e3fdf to 0x15f3218, (0xf239 bytes) 2024-09-13T04:58:21.832 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.832 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.849 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:21.844+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:21.864 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:21.865 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:21.869 INFO:teuthology.orchestra.run.smithi001.stdout:917 write 0xafc7c59 thru 0xafd3ceb (0xc093 bytes) 2024-09-13T04:58:21.877 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.878 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:21.881 INFO:teuthology.orchestra.run.smithi001.stdout:919 trunc from 0xafd3cec to 0xc9341c9 2024-09-13T04:58:21.893 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.894 INFO:teuthology.orchestra.run.smithi001.stdout:922 writesame 0x82b5bf thru 0x833036 data_size 0x4(0x7a78 bytes) 2024-09-13T04:58:21.909 INFO:teuthology.orchestra.run.smithi001.stdout:926 punch from 0x410872e to 0x410e228, (0x5afa bytes) 2024-09-13T04:58:21.910 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:21.928 INFO:teuthology.orchestra.run.smithi001.stdout:932 write 0xe4733a8 thru 0xe47500b (0x1c64 bytes) 2024-09-13T04:58:21.934 INFO:teuthology.orchestra.run.smithi001.stdout:933 read 0xdc93233 thru 0xdc95f98 (0x2d66 bytes) 2024-09-13T04:58:21.935 INFO:teuthology.orchestra.run.smithi001.stdout:934 write 0xb06931a thru 0xb073cb8 (0xa99f bytes) 2024-09-13T04:58:21.944 INFO:teuthology.orchestra.run.smithi001.stdout:935 read 0x35cbef2 thru 0x35d8357 (0xc466 bytes) 2024-09-13T04:58:21.944 INFO:teuthology.orchestra.run.smithi001.stdout:936 write 0xa570a7c thru 0xa5724e4 (0x1a69 bytes) 2024-09-13T04:58:21.947 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:21.944+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:21.986 INFO:teuthology.orchestra.run.smithi001.stdout:937 trunc from 0xe47500c to 0xde1758 2024-09-13T04:58:22.002 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:22.002 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:22.014 INFO:teuthology.orchestra.run.smithi001.stdout:939 trunc from 0xde1758 to 0xcc8283 2024-09-13T04:58:22.017 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:22.048 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:22.044+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:22.064 INFO:teuthology.orchestra.run.smithi001.stdout:940 write 0x98cd8f6 thru 0x98cf5d5 (0x1ce0 bytes) 2024-09-13T04:58:22.069 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:22.069 INFO:teuthology.orchestra.run.smithi001.stdout:942 read 0x3e469ef thru 0x3e565ec (0xfbfe bytes) 2024-09-13T04:58:22.079 INFO:teuthology.orchestra.run.smithi001.stdout:943 trunc from 0x98cf5d6 to 0x5bb61c6 2024-09-13T04:58:22.100 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:22.100 INFO:teuthology.orchestra.run.smithi001.stdout:947 trunc from 0x5bb61c6 to 0x5ede867 2024-09-13T04:58:22.140 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:22.140 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:22.144 INFO:teuthology.orchestra.run.smithi001.stdout:948 writesame 0xdcb4059 thru 0xdcb41c3 data_size 0x3(0x16b bytes) 2024-09-13T04:58:22.149 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:22.148+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:22.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:22.156 INFO:teuthology.orchestra.run.smithi001.stdout:949 trunc from 0xdcb41c4 to 0x9b6ff1a 2024-09-13T04:58:22.170 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:22.170 INFO:teuthology.orchestra.run.smithi001.stdout:953 write 0x741f993 thru 0x742c97b (0xcfe9 bytes) 2024-09-13T04:58:22.182 INFO:teuthology.orchestra.run.smithi001.stdout:954 read 0x6ea8a6 thru 0x6ee23b (0x3996 bytes) 2024-09-13T04:58:22.183 INFO:teuthology.orchestra.run.smithi001.stdout:956 writesame 0x13cebac thru 0x13d2025 data_size 0x3(0x347a bytes) 2024-09-13T04:58:22.195 INFO:teuthology.orchestra.run.smithi001.stdout:957 write 0x417dc7a thru 0x418c496 (0xe81d bytes) 2024-09-13T04:58:22.236 INFO:teuthology.orchestra.run.smithi001.stdout:958 writesame 0xcc5942e thru 0xcc595e6 data_size 0x9(0x1b9 bytes) 2024-09-13T04:58:22.249 INFO:teuthology.orchestra.run.smithi001.stdout:961 write 0xd18238f thru 0xd185d92 (0x3a04 bytes) 2024-09-13T04:58:22.250 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:22.248+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:22.256 INFO:teuthology.orchestra.run.smithi001.stdout:962 read 0x59883b1 thru 0x599063c (0x828c bytes) 2024-09-13T04:58:22.257 INFO:teuthology.orchestra.run.smithi001.stdout:964 punch from 0x2851b86 to 0x2856cee, (0x5168 bytes) 2024-09-13T04:58:22.257 INFO:teuthology.orchestra.run.smithi001.stdout:966 read 0x86ae72e thru 0x86b416f (0x5a42 bytes) 2024-09-13T04:58:22.257 INFO:teuthology.orchestra.run.smithi001.stdout:967 writesame 0xc856889 thru 0xc8574a2 data_size 0x1(0xc1a bytes) 2024-09-13T04:58:22.267 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:22.278 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:22.278 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:22.283 INFO:teuthology.orchestra.run.smithi001.stdout:969 trunc from 0xd185d93 to 0x710f778 2024-09-13T04:58:22.293 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:22.338 INFO:teuthology.orchestra.run.smithi001.stdout:970 writesame 0xd00c2d5 thru 0xd01af83 data_size 0x13(0xecaf bytes) 2024-09-13T04:58:22.338 INFO:teuthology.orchestra.run.smithi001.stdout:971 read 0x440b458 thru 0x440bdd8 (0x981 bytes) 2024-09-13T04:58:22.339 INFO:teuthology.orchestra.run.smithi001.stdout:972 write 0x58ab059 thru 0x58b1d67 (0x6d0f bytes) 2024-09-13T04:58:22.346 INFO:teuthology.orchestra.run.smithi001.stdout:973 write 0x826f8ba thru 0x8272ed0 (0x3617 bytes) 2024-09-13T04:58:22.351 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:22.348+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:22.353 INFO:teuthology.orchestra.run.smithi001.stdout:975 read 0x623746a thru 0x62457c3 (0xe35a bytes) 2024-09-13T04:58:22.353 INFO:teuthology.orchestra.run.smithi001.stdout:976 read 0x3967e4c thru 0x3971ea9 (0xa05e bytes) 2024-09-13T04:58:22.353 INFO:teuthology.orchestra.run.smithi001.stdout:977 writesame 0xaecb35a thru 0xaed87c3 data_size 0x72(0xd46a bytes) 2024-09-13T04:58:22.360 INFO:teuthology.orchestra.run.smithi001.stdout:978 punch from 0x8d0453f to 0x8d0734d, (0x2e0e bytes) 2024-09-13T04:58:22.360 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:22.360 INFO:teuthology.orchestra.run.smithi001.stdout:981 read 0x2574ba4 thru 0x257f41c (0xa879 bytes) 2024-09-13T04:58:22.360 INFO:teuthology.orchestra.run.smithi001.stdout:982 writesame 0xb27674b thru 0xb2852d8 data_size 0xb(0xeb8e bytes) 2024-09-13T04:58:22.374 INFO:teuthology.orchestra.run.smithi001.stdout:983 read 0x8054a8f thru 0x806113b (0xc6ad bytes) 2024-09-13T04:58:22.375 INFO:teuthology.orchestra.run.smithi001.stdout:984 clone 7 order 22 su 2097152 sc 15 2024-09-13T04:58:22.418 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:22.418 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:22.433 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:22.480 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:22.480+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:22.553 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:22.552+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:22.555 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:22.555 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:22.569 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:22.654 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:22.652+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:22.737 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:22.737 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:22.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:22.755 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:22.752+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:22.856 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:22.856+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:22.872 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:22.872 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:22.886 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:22.957 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:22.956+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:23.048 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:23.048 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:23.058 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:23.056+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:23.063 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:23.159 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:23.156+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:23.184 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:23.184 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:23.198 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:23.260 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:23.256+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:23.325 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:58:23.326 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:58:23.326 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T04:58:23.326 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-13T04:58:23.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:58:23.361 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:23.360+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:23.365 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:23.365 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:23.380 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:23.409 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:23.468 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:23.468+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:23.529 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-13T04:58:23.542 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:23.542 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:23.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:23.577 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:23.576+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:23.583 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:23.583 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:23.602 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:23.603 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:23.623 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:23.624 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:23.639 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-13T04:58:23.643 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:23.643 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:23.664 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:23.660+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:23.668 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:23.669 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:23.694 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:23.695 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:23.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T04:58:23.739 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:23.766 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:23.766 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:32931 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.706749+0000", 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.042685397999999999, 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042716276999999997, 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 32931 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706749+0000", 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706749+0000", 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706748+0000", 2024-09-13T04:58:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706758+0000", 2024-09-13T04:58:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.054e-06 2024-09-13T04:58:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706764+0000", 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.922e-06 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706771+0000", 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3640000000000004e-06 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33044 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718535+0000", 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030899452000000001, 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031006355999999999, 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33044 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718535+0000", 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718535+0000", 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718534+0000", 2024-09-13T04:58:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718536+0000", 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9089999999999998e-06 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718542+0000", 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5219999999999998e-06 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718546+0000", 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3830000000000002e-06 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030244483999999999, 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030416644999999999, 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:58:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33264 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.725980+0000", 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.02345421, 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023693617, 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33264 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.725980+0000", 2024-09-13T04:58:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.725980+0000", 2024-09-13T04:58:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.725980+0000", 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.725984+0000", 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3680000000000004e-06 2024-09-13T04:58:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.725991+0000", 2024-09-13T04:58:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1930000000000004e-06 2024-09-13T04:58:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726005+0000", 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3813e-05 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33296 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729697+0000", 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019736805999999999, 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020042378, 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33296 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729697+0000", 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729697+0000", 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729697+0000", 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729700+0000", 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8399999999999999e-06 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729707+0000", 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8569999999999997e-06 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729712+0000", 2024-09-13T04:58:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0780000000000003e-06 2024-09-13T04:58:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33328 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733391+0000", 2024-09-13T04:58:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.016043640000000001, 2024-09-13T04:58:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016413157000000001, 2024-09-13T04:58:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33328 2024-09-13T04:58:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733391+0000", 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733391+0000", 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733390+0000", 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733393+0000", 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6979999999999998e-06 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733400+0000", 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6980000000000001e-06 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733405+0000", 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33364 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738743+0000", 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010691541000000001, 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011123107, 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33364 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738743+0000", 2024-09-13T04:58:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738743+0000", 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738742+0000", 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738750+0000", 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8839999999999994e-06 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738757+0000", 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7819999999999996e-06 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738762+0000", 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:32932 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.706777+0000", 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.042657390000000003, 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043148104999999999, 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 32932 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706777+0000", 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706777+0000", 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706776+0000", 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706779+0000", 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4030000000000001e-06 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706785+0000", 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2790000000000004e-06 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706796+0000", 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1075e-05 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33046 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718551+0000", 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030883689999999998, 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031442592999999998, 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33046 2024-09-13T04:58:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718551+0000", 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718551+0000", 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718550+0000", 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718552+0000", 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1900000000000002e-06 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718558+0000", 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5690000000000004e-06 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718562+0000", 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3919999999999996e-06 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33265 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.726011+0000", 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.023423282, 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024069314000000001, 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33265 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726011+0000", 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726011+0000", 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726010+0000", 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726014+0000", 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3620000000000002e-06 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726020+0000", 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5540000000000003e-06 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726026+0000", 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2120000000000002e-06 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33297 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729716+0000", 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019717946, 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020412144, 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33297 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729716+0000", 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729716+0000", 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729716+0000", 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729718+0000", 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4260000000000002e-06 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729725+0000", 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4520000000000002e-06 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729729+0000", 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.775e-06 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33329 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733409+0000", 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.016025055, 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016752257999999999, 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33329 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733409+0000", 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733409+0000", 2024-09-13T04:58:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733409+0000", 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733411+0000", 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2970000000000002e-06 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733417+0000", 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9660000000000001e-06 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733422+0000", 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9540000000000003e-06 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33366 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738767+0000", 2024-09-13T04:58:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010667685999999999, 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011459596000000001, 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33366 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738767+0000", 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738767+0000", 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738766+0000", 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738769+0000", 2024-09-13T04:58:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6390000000000001e-06 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738775+0000", 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4200000000000004e-06 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738780+0000", 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6190000000000002e-06 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:32933 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.706804+0000", 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.042630596, 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043515308000000003, 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 32933 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706804+0000", 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706804+0000", 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706803+0000", 2024-09-13T04:58:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706806+0000", 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4820000000000001e-06 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706813+0000", 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660000000000002e-06 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706818+0000", 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2070000000000003e-06 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33048 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718566+0000", 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030867800000000001, 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031813582, 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33048 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718566+0000", 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718566+0000", 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718566+0000", 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718568+0000", 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1909999999999999e-06 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718574+0000", 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0589999999999999e-06 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718579+0000", 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6410000000000001e-06 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33266 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.726460+0000", 2024-09-13T04:58:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.022973773999999999, 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023973477, 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33266 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726460+0000", 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726460+0000", 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726460+0000", 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726465+0000", 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7450000000000003e-06 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726477+0000", 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.207e-05 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726484+0000", 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2479999999999999e-06 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33298 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019700641000000001, 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020785674000000001, 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33298 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729733+0000", 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729735+0000", 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.875e-06 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729741+0000", 2024-09-13T04:58:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.417e-06 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729754+0000", 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3811999999999999e-05 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33330 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733426+0000", 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.016007912999999999, 2024-09-13T04:58:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017126144999999999, 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33330 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733426+0000", 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733426+0000", 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733426+0000", 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733434+0000", 2024-09-13T04:58:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8499999999999994e-06 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733441+0000", 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9789999999999996e-06 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733446+0000", 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0860000000000001e-06 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33385 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738784+0000", 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010650365, 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011803855, 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33385 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738784+0000", 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738784+0000", 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738784+0000", 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738786+0000", 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5129999999999999e-06 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738792+0000", 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2539999999999998e-06 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738797+0000", 2024-09-13T04:58:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7500000000000003e-06 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:32934 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.706832+0000", 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.042602503, 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043832971999999998, 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 32934 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706832+0000", 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706832+0000", 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706831+0000", 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706834+0000", 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6070000000000002e-06 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706840+0000", 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3670000000000002e-06 2024-09-13T04:58:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706845+0000", 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4320000000000001e-06 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33050 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718583+0000", 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030850776999999999, 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032143691000000002, 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33050 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718583+0000", 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718583+0000", 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718583+0000", 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718585+0000", 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1179999999999999e-06 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718591+0000", 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0599999999999996e-06 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718596+0000", 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5290000000000002e-06 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33267 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.726490+0000", 2024-09-13T04:58:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.022944688000000001, 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024299798000000001, 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33267 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726490+0000", 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726490+0000", 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726489+0000", 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.726492+0000", 2024-09-13T04:58:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1640000000000001e-06 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727046+0000", 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00055399099999999999 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727053+0000", 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.973e-06 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33299 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729760+0000", 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019674289000000001, 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021062821999999998, 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33299 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729760+0000", 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729760+0000", 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729760+0000", 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729762+0000", 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7810000000000001e-06 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729769+0000", 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1629999999999999e-06 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729774+0000", 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6909999999999996e-06 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33331 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733451+0000", 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015983468000000001, 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017417750999999999, 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33331 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733451+0000", 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733451+0000", 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733450+0000", 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733453+0000", 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3760000000000002e-06 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733458+0000", 2024-09-13T04:58:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8549999999999998e-06 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733472+0000", 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3967e-05 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33387 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.742337+0000", 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0070974519999999998, 2024-09-13T04:58:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085947109999999997, 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33387 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.742337+0000", 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.742337+0000", 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738801+0000", 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9964638 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.742342+0000", 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00354141 2024-09-13T04:58:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.742355+0000", 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2996999999999999e-05 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.742363+0000", 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.994e-06 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33006 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.706849+0000", 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.042585111000000002, 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044144969999999999, 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33006 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706849+0000", 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706849+0000", 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706849+0000", 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706856+0000", 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6799999999999993e-06 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706863+0000", 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2759999999999997e-06 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706874+0000", 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1559999999999999e-05 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33190 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718600+0000", 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030834324999999999, 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032449278999999998, 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33190 2024-09-13T04:58:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718600+0000", 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718600+0000", 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718600+0000", 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718608+0000", 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2169999999999994e-06 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718615+0000", 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0079999999999996e-06 2024-09-13T04:58:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718620+0000", 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9049999999999996e-06 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33268 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.727859+0000", 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021575034, 2024-09-13T04:58:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023223097000000002, 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33268 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727859+0000", 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727859+0000", 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727858+0000", 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727864+0000", 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1340000000000003e-06 2024-09-13T04:58:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727874+0000", 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046e-05 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727882+0000", 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.157e-06 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019655702000000001, 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021340173, 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:58:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33332 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733477+0000", 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015956883000000002, 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017674835999999999, 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33332 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733477+0000", 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733477+0000", 2024-09-13T04:58:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733477+0000", 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733480+0000", 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7599999999999998e-06 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733487+0000", 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.985e-06 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733492+0000", 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1939999999999997e-06 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33725 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.749876+0000", 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.999558, 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001307674, 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33725 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749876+0000", 2024-09-13T04:58:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749876+0000", 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749875+0000", 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749883+0000", 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6980000000000004e-06 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749895+0000", 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1361e-05 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749903+0000", 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6419999999999996e-06 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33008 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.706879+0000", 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.042555388, 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044338990000000002, 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33008 2024-09-13T04:58:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706879+0000", 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706879+0000", 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706878+0000", 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706881+0000", 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706887+0000", 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.387e-06 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706892+0000", 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8239999999999999e-06 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33191 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718624+0000", 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030809758, 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032646504999999999, 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33191 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718624+0000", 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718624+0000", 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718624+0000", 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718626+0000", 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4040000000000002e-06 2024-09-13T04:58:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718633+0000", 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.178e-06 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718643+0000", 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0166000000000001e-05 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33269 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.727888+0000", 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021546362999999999, 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023430847000000001, 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33269 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727888+0000", 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727888+0000", 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727887+0000", 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727896+0000", 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1970000000000002e-06 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727904+0000", 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9640000000000003e-06 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727910+0000", 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4909999999999996e-06 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019631217999999999, 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021577298000000002, 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:58:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33333 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733496+0000", 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015937758999999999, 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981433000000002, 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33333 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733496+0000", 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733496+0000", 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733496+0000", 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733498+0000", 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4789999999999999e-06 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733505+0000", 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1700000000000002e-06 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733509+0000", 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6129999999999997e-06 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33727 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.749911+0000", 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9995236, 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001617519, 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33727 2024-09-13T04:58:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749911+0000", 2024-09-13T04:58:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749911+0000", 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749910+0000", 2024-09-13T04:58:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749913+0000", 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9340000000000002e-06 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749920+0000", 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4459999999999998e-06 2024-09-13T04:58:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749925+0000", 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2889999999999997e-06 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33010 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.706896+0000", 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.042537885999999997, 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044667837000000002, 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33010 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706896+0000", 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706896+0000", 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706896+0000", 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706898+0000", 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706904+0000", 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7049999999999996e-06 2024-09-13T04:58:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706908+0000", 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4340000000000002e-06 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33192 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718648+0000", 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030786507000000001, 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032971008000000003, 2024-09-13T04:58:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33192 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718648+0000", 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718648+0000", 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718647+0000", 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718650+0000", 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4889999999999998e-06 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718656+0000", 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4529999999999999e-06 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718661+0000", 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7690000000000004e-06 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33270 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.727915+0000", 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021519450999999998, 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023757093999999999, 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33270 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727915+0000", 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727915+0000", 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727914+0000", 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727917+0000", 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7199999999999998e-06 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727923+0000", 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.353e-06 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727928+0000", 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8829999999999997e-06 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019615250000000001, 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021885692000000002, 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:58:23.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:58:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33334 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733515+0000", 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015919289999999999, 2024-09-13T04:58:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018221108999999999, 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33334 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733515+0000", 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733515+0000", 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733515+0000", 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733517+0000", 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2340000000000001e-06 2024-09-13T04:58:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733523+0000", 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2979999999999997e-06 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733535+0000", 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2148e-05 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33730 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.749930+0000", 2024-09-13T04:58:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9995046, 2024-09-13T04:58:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018397649999999999, 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33730 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749930+0000", 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749930+0000", 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749929+0000", 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749932+0000", 2024-09-13T04:58:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.295e-06 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749938+0000", 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.998e-06 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749942+0000", 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6399999999999996e-06 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33012 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.706912+0000", 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.042522113, 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044888465000000002, 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33012 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706912+0000", 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706912+0000", 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706912+0000", 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706914+0000", 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3590000000000002e-06 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706920+0000", 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.891e-06 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706925+0000", 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6519999999999997e-06 2024-09-13T04:58:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33193 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718665+0000", 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030768917999999999, 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033188779000000002, 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33193 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718665+0000", 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718665+0000", 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718665+0000", 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718667+0000", 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718673+0000", 2024-09-13T04:58:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-06 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718678+0000", 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33271 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.727933+0000", 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021501505000000001, 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023987518999999999, 2024-09-13T04:58:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33271 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727933+0000", 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727933+0000", 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727932+0000", 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727935+0000", 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.655e-06 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727941+0000", 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.421e-06 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727946+0000", 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9180000000000002e-06 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33303 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729836+0000", 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019598063999999998, 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022120440000000002, 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33303 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729836+0000", 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729836+0000", 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729836+0000", 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729838+0000", 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1019999999999999e-06 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729844+0000", 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9950000000000002e-06 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729848+0000", 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2969999999999997e-06 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33335 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733542+0000", 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015892316, 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018446002999999999, 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33335 2024-09-13T04:58:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733542+0000", 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733542+0000", 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733541+0000", 2024-09-13T04:58:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733544+0000", 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6520000000000002e-06 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733551+0000", 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6309999999999997e-06 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733556+0000", 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.933e-06 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33732 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.749946+0000", 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9994879, 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020750230000000001, 2024-09-13T04:58:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33732 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749946+0000", 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749946+0000", 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749946+0000", 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749948+0000", 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.379e-06 2024-09-13T04:58:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749954+0000", 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9710000000000001e-06 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749967+0000", 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2724999999999999e-05 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33014 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.707835+0000", 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041599230000000001, 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044217448999999999, 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33014 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707835+0000", 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707835+0000", 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.706928+0000", 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9990931 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707839+0000", 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00091110900000000001 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707849+0000", 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5510000000000005e-06 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707855+0000", 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6780000000000003e-06 2024-09-13T04:58:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33194 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718682+0000", 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030752044999999999, 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033410858000000002, 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33194 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718682+0000", 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718682+0000", 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718682+0000", 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718689+0000", 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7479999999999996e-06 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718695+0000", 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8820000000000003e-06 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718708+0000", 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2515e-05 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33272 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.727951+0000", 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021483516000000001, 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024204327000000001, 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33272 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727951+0000", 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727951+0000", 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727950+0000", 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727952+0000", 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.962e-06 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727958+0000", 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8660000000000002e-06 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727963+0000", 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9010000000000002e-06 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:58:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019581978999999999, 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022366296000000001, 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:58:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33336 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733560+0000", 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.01587423, 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018718530000000001, 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33336 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733560+0000", 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733560+0000", 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733560+0000", 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733568+0000", 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2430000000000005e-06 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733575+0000", 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4580000000000004e-06 2024-09-13T04:58:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733581+0000", 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.152e-06 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33735 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.749972+0000", 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9994621, 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002370058, 2024-09-13T04:58:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33735 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749972+0000", 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749972+0000", 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749972+0000", 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749974+0000", 2024-09-13T04:58:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8830000000000002e-06 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749981+0000", 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5849999999999996e-06 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749986+0000", 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9219999999999996e-06 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33016 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.707862+0000", 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041572460999999998, 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044547246999999998, 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33016 2024-09-13T04:58:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707862+0000", 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707862+0000", 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707861+0000", 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707864+0000", 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8590000000000001e-06 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707871+0000", 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4840000000000001e-06 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707875+0000", 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7060000000000003e-06 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33195 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718713+0000", 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030721235, 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033758927000000001, 2024-09-13T04:58:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33195 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718713+0000", 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718713+0000", 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718713+0000", 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718715+0000", 2024-09-13T04:58:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4329999999999998e-06 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718722+0000", 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4989999999999999e-06 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718726+0000", 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6990000000000002e-06 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33273 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.727968+0000", 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021466645999999999, 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024571118999999999, 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33273 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727968+0000", 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727968+0000", 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727967+0000", 2024-09-13T04:58:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.727969+0000", 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2029999999999999e-06 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728213+0000", 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000243966 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728221+0000", 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1400000000000002e-06 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33305 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729875+0000", 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019558833000000001, 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022725097, 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33305 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729875+0000", 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729875+0000", 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729875+0000", 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729877+0000", 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1179999999999999e-06 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729890+0000", 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3244999999999999e-05 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730934+0000", 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001043616 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33337 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.974 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733586+0000", 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015848643999999999, 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019085478999999999, 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33337 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733586+0000", 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733586+0000", 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733585+0000", 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733588+0000", 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6800000000000002e-06 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733594+0000", 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1029999999999998e-06 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733598+0000", 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4780000000000002e-06 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.977 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33739 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.749990+0000", 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.999444, 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027459670000000002, 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33739 2024-09-13T04:58:23.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749990+0000", 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749990+0000", 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749990+0000", 2024-09-13T04:58:23.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749992+0000", 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4360000000000001e-06 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.749998+0000", 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0129999999999999e-06 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750003+0000", 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4880000000000001e-06 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33017 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.707880+0000", 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.04155474, 2024-09-13T04:58:23.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044924929000000002, 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33017 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707880+0000", 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707880+0000", 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707879+0000", 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707881+0000", 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1519999999999999e-06 2024-09-13T04:58:23.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707888+0000", 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707900+0000", 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2503e-05 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.984 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33196 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718731+0000", 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030703372999999999, 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034130738000000001, 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33196 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718731+0000", 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718731+0000", 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718730+0000", 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718733+0000", 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.221e-06 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718739+0000", 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1829999999999999e-06 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718744+0000", 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.987 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33274 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728228+0000", 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021205979999999999, 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024671183999999999, 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33274 2024-09-13T04:58:23.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728228+0000", 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728228+0000", 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728228+0000", 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728232+0000", 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1590000000000003e-06 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728240+0000", 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4980000000000006e-06 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728245+0000", 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1610000000000002e-06 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33306 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.730948+0000", 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018485753000000001, 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022006042999999999, 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33306 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730948+0000", 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730948+0000", 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730948+0000", 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730957+0000", 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0750000000000004e-06 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730968+0000", 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1229999999999999e-05 2024-09-13T04:58:23.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730975+0000", 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3209999999999999e-06 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33338 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733603+0000", 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015831669999999999, 2024-09-13T04:58:23.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019398459999999999, 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33338 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733603+0000", 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733603+0000", 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733602+0000", 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733604+0000", 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.255e-06 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733611+0000", 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2979999999999997e-06 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:23.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733615+0000", 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5600000000000004e-06 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33741 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750007+0000", 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9994273, 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030306159999999999, 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:23.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33741 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750007+0000", 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:23.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750007+0000", 2024-09-13T04:58:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750006+0000", 2024-09-13T04:58:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750009+0000", 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1380000000000001e-06 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750014+0000", 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6740000000000002e-06 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750025+0000", 2024-09-13T04:58:24.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0703999999999999e-05 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33018 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.707906+0000", 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041528382000000003, 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045165548999999999, 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33018 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707906+0000", 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707906+0000", 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707905+0000", 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707908+0000", 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7460000000000001e-06 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707915+0000", 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6919999999999996e-06 2024-09-13T04:58:24.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707920+0000", 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8160000000000001e-06 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33197 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718748+0000", 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030686086000000001, 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034385908, 2024-09-13T04:58:24.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33197 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718748+0000", 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718748+0000", 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718748+0000", 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718750+0000", 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2060000000000001e-06 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718756+0000", 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718761+0000", 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7430000000000002e-06 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33275 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.008 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728250+0000", 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021184149999999999, 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024946680999999998, 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33275 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728250+0000", 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728250+0000", 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728250+0000", 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728252+0000", 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4329999999999998e-06 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728259+0000", 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5440000000000004e-06 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728264+0000", 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0459999999999996e-06 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33307 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018453414000000001, 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022366416, 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33307 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730980+0000", 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730983+0000", 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5359999999999999e-06 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730989+0000", 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730994+0000", 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8040000000000001e-06 2024-09-13T04:58:24.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33339 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733620+0000", 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015814730999999999, 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019794517000000001, 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33339 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733620+0000", 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733620+0000", 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733619+0000", 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733621+0000", 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2469999999999998e-06 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733627+0000", 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0920000000000003e-06 2024-09-13T04:58:24.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733632+0000", 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4710000000000001e-06 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33742 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750029+0000", 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9994054, 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034519780000000001, 2024-09-13T04:58:24.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33742 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750029+0000", 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750029+0000", 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750029+0000", 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750031+0000", 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4269999999999998e-06 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750037+0000", 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7660000000000004e-06 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750041+0000", 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.228e-06 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.021 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33019 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.707924+0000", 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041510345999999997, 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045625009000000001, 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33019 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707924+0000", 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707924+0000", 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707924+0000", 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707933+0000", 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1030000000000008e-06 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707939+0000", 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1569999999999997e-06 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707943+0000", 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6530000000000002e-06 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33198 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718765+0000", 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030669049, 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034851604000000001, 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33198 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.025 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718765+0000", 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718765+0000", 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718765+0000", 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718767+0000", 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1409999999999999e-06 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718773+0000", 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9909999999999999e-06 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718778+0000", 2024-09-13T04:58:24.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6120000000000001e-06 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33276 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728269+0000", 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021165571000000001, 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025409709999999999, 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33276 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728269+0000", 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728269+0000", 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728268+0000", 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728283+0000", 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4487e-05 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728290+0000", 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7479999999999993e-06 2024-09-13T04:58:24.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728296+0000", 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5389999999999998e-06 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33308 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.730999+0000", 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018435528, 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022744944, 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33308 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730999+0000", 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730999+0000", 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730998+0000", 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731001+0000", 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2340000000000001e-06 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731007+0000", 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.19e-06 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731011+0000", 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4580000000000003e-06 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33340 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733636+0000", 2024-09-13T04:58:24.034 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015798091, 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020168243999999998, 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33340 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733636+0000", 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733636+0000", 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733636+0000", 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733638+0000", 2024-09-13T04:58:24.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1519999999999999e-06 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733644+0000", 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0560000000000001e-06 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733656+0000", 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2156e-05 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33744 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750045+0000", 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9993896, 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038193250000000002, 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33744 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750045+0000", 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750045+0000", 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750044+0000", 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750051+0000", 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3910000000000003e-06 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750057+0000", 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8969999999999996e-06 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750061+0000", 2024-09-13T04:58:24.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2869999999999998e-06 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33020 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.707950+0000", 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041484162999999998, 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045978663000000003, 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33020 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707950+0000", 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707950+0000", 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707950+0000", 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707952+0000", 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5900000000000002e-06 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707958+0000", 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1349999999999997e-06 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707963+0000", 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.814e-06 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33199 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718782+0000", 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030652441999999998, 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035207479, 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33199 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718782+0000", 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718782+0000", 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718781+0000", 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718784+0000", 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.063e-06 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718789+0000", 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7620000000000001e-06 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718793+0000", 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0899999999999998e-06 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33277 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728301+0000", 2024-09-13T04:58:24.047 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021133430000000002, 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025756998999999999, 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33277 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728301+0000", 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728301+0000", 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728300+0000", 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728303+0000", 2024-09-13T04:58:24.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7920000000000001e-06 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728309+0000", 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2659999999999998e-06 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728322+0000", 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2571999999999999e-05 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018418447000000001, 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023101465000000002, 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:58:24.053 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33341 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733665+0000", 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015769663999999999, 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020519929999999999, 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33341 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733665+0000", 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733665+0000", 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733664+0000", 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733667+0000", 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7999999999999999e-06 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733674+0000", 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5320000000000001e-06 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733679+0000", 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0710000000000002e-06 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33746 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750065+0000", 2024-09-13T04:58:24.057 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9993691, 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041899119999999996, 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33746 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750065+0000", 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750065+0000", 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750065+0000", 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750067+0000", 2024-09-13T04:58:24.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1239999999999999e-06 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750072+0000", 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.344e-06 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750076+0000", 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2699999999999998e-06 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33021 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.707967+0000", 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041466940000000001, 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046354755999999997, 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33021 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707967+0000", 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707967+0000", 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707967+0000", 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707969+0000", 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1019999999999999e-06 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707975+0000", 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6910000000000002e-06 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707979+0000", 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.544e-06 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.063 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33200 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718798+0000", 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030636527, 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035589849, 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33200 2024-09-13T04:58:24.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718798+0000", 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718798+0000", 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718797+0000", 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718799+0000", 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1129999999999999e-06 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718805+0000", 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5520000000000004e-06 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.067 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:23.764+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:24.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718815+0000", 2024-09-13T04:58:24.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0319999999999999e-05 2024-09-13T04:58:24.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33278 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728328+0000", 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021106640999999999, 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026129968, 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33278 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728328+0000", 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728328+0000", 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728327+0000", 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728330+0000", 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8380000000000002e-06 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728349+0000", 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9426999999999999e-05 2024-09-13T04:58:24.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728355+0000", 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8209999999999998e-06 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33310 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731032+0000", 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018402602000000001, 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023488798000000002, 2024-09-13T04:58:24.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33310 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731032+0000", 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731032+0000", 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731031+0000", 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731033+0000", 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1239999999999999e-06 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731039+0000", 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8730000000000004e-06 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731044+0000", 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5310000000000003e-06 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33342 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.075 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733684+0000", 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015750424999999998, 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020905628999999998, 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33342 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733684+0000", 2024-09-13T04:58:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733684+0000", 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733683+0000", 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733690+0000", 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6719999999999998e-06 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733696+0000", 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3369999999999996e-06 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733701+0000", 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7500000000000003e-06 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33748 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750080+0000", 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9993544, 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004573326, 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33748 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750080+0000", 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750080+0000", 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750080+0000", 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750082+0000", 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0049999999999999e-06 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750087+0000", 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4519999999999996e-06 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750091+0000", 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.16e-06 2024-09-13T04:58:24.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33022 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.707984+0000", 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041449991999999998, 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046732325999999998, 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33022 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707984+0000", 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707984+0000", 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707984+0000", 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707986+0000", 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1229999999999998e-06 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707992+0000", 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8370000000000002e-06 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.707996+0000", 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6059999999999996e-06 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33201 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718821+0000", 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030613728999999999, 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035963643000000003, 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33201 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718821+0000", 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718821+0000", 2024-09-13T04:58:24.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718820+0000", 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718823+0000", 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5440000000000001e-06 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718829+0000", 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1650000000000003e-06 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718833+0000", 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4299999999999999e-06 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33279 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728360+0000", 2024-09-13T04:58:24.088 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021074117, 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026486769, 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33279 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728360+0000", 2024-09-13T04:58:24.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728360+0000", 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728360+0000", 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728362+0000", 2024-09-13T04:58:24.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5859999999999999e-06 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728369+0000", 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5549999999999999e-06 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728374+0000", 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7369999999999997e-06 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.018386019, 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023865599000000001, 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.094 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33343 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733707+0000", 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015727508000000001, 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021269571000000001, 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33343 2024-09-13T04:58:24.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733707+0000", 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733707+0000", 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733706+0000", 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733709+0000", 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5519999999999999e-06 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733715+0000", 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3380000000000001e-06 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733720+0000", 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.955e-06 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33750 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750095+0000", 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9993391, 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049491329999999997, 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33750 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750095+0000", 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750095+0000", 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750095+0000", 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750097+0000", 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9149999999999999e-06 2024-09-13T04:58:24.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750102+0000", 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4759999999999997e-06 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750107+0000", 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3869999999999997e-06 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33023 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.708024+0000", 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041410084, 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047086208999999997, 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33023 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708024+0000", 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708024+0000", 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708024+0000", 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708032+0000", 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9969999999999998e-06 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708040+0000", 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8050000000000007e-06 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708045+0000", 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5230000000000003e-06 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.104 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33202 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718838+0000", 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030596732000000001, 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036338486000000003, 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33202 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.105 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718838+0000", 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718838+0000", 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718837+0000", 2024-09-13T04:58:24.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718847+0000", 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3770000000000002e-06 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718853+0000", 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8360000000000003e-06 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718858+0000", 2024-09-13T04:58:24.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.904e-06 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33280 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728378+0000", 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.021056452999999999, 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026859244000000001, 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33280 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728378+0000", 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728378+0000", 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728377+0000", 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728380+0000", 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1840000000000002e-06 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728386+0000", 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728398+0000", 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2149e-05 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33312 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731671+0000", 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017763725000000001, 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023634367, 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33312 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731671+0000", 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731671+0000", 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731670+0000", 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731683+0000", 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3307e-05 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731696+0000", 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3274e-05 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731704+0000", 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.593e-06 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33344 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733724+0000", 2024-09-13T04:58:24.114 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015709873999999999, 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021642328999999998, 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33344 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733724+0000", 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733724+0000", 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733724+0000", 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733726+0000", 2024-09-13T04:58:24.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733732+0000", 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8749999999999997e-06 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733737+0000", 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.566e-06 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33752 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750111+0000", 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9993238, 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053236350000000002, 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33752 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.118 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750111+0000", 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750111+0000", 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750110+0000", 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750116+0000", 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0170000000000002e-06 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750122+0000", 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9359999999999996e-06 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750134+0000", 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1552000000000001e-05 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33024 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.708052+0000", 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.041382580000000002, 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047445690999999998, 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.121 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33024 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708052+0000", 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708052+0000", 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708051+0000", 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708054+0000", 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7149999999999998e-06 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.708067+0000", 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3468999999999999e-05 2024-09-13T04:58:24.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710623+0000", 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025553360000000001 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33203 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718892+0000", 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030541967, 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036670386999999999, 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33203 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718892+0000", 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718892+0000", 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718892+0000", 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718905+0000", 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3200000000000001e-05 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718914+0000", 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3209999999999994e-06 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718920+0000", 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7270000000000004e-06 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33281 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.127 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728962+0000", 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020472495, 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026668536, 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33281 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728962+0000", 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728962+0000", 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728403+0000", 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9994411 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728971+0000", 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00056842800000000005 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728982+0000", 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1058e-05 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728989+0000", 2024-09-13T04:58:24.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1149999999999996e-06 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017724386000000002, 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023986398999999999, 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.131 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:58:24.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33345 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733741+0000", 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.015693563000000001, 2024-09-13T04:58:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022025327000000001, 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33345 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733741+0000", 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733741+0000", 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733740+0000", 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733743+0000", 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733749+0000", 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9819999999999996e-06 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.734796+0000", 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010472509999999999 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.137 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33754 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750138+0000", 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9992962, 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056907590000000001, 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33754 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750138+0000", 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750138+0000", 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750138+0000", 2024-09-13T04:58:24.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750140+0000", 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4540000000000001e-06 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750146+0000", 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6189999999999999e-06 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750150+0000", 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2860000000000001e-06 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33025 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.710632+0000", 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.038801892999999997, 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045263783000000002, 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33025 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710632+0000", 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710632+0000", 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710632+0000", 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710636+0000", 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710645+0000", 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6179999999999995e-06 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710651+0000", 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3799999999999999e-06 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33204 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718928+0000", 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030506064999999999, 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037030896000000001, 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33204 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718928+0000", 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718928+0000", 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718928+0000", 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718931+0000", 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7350000000000001e-06 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718937+0000", 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718942+0000", 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8940000000000001e-06 2024-09-13T04:58:24.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33282 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.728997+0000", 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020437496999999999, 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027029095, 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33282 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728997+0000", 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728997+0000", 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728996+0000", 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.728999+0000", 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2780000000000002e-06 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729007+0000", 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2350000000000002e-06 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729012+0000", 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.604e-06 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33314 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731738+0000", 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017696351999999999, 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024349981999999999, 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33314 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731738+0000", 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731738+0000", 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731737+0000", 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731740+0000", 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.852e-06 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731748+0000", 2024-09-13T04:58:24.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3320000000000003e-06 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731753+0000", 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0010000000000001e-06 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33346 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.734812+0000", 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.014622405999999999, 2024-09-13T04:58:24.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021343371999999999, 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33346 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.734812+0000", 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.734812+0000", 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.734811+0000", 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.734816+0000", 2024-09-13T04:58:24.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8770000000000001e-06 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.734826+0000", 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0018e-05 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.734833+0000", 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1069999999999999e-06 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33757 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750154+0000", 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9992805, 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060656010000000003, 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33757 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750154+0000", 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750154+0000", 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750154+0000", 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750156+0000", 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.99e-06 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750161+0000", 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4029999999999998e-06 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750165+0000", 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2139999999999998e-06 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.160 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33026 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.710657+0000", 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.038777170999999999, 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045632297000000002, 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33026 2024-09-13T04:58:24.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710657+0000", 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710657+0000", 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710657+0000", 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710659+0000", 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.407e-06 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710664+0000", 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4229999999999996e-06 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710678+0000", 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33205 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718946+0000", 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030488132000000001, 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037411613000000003, 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.164 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33205 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718946+0000", 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718946+0000", 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718946+0000", 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718948+0000", 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.227e-06 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718954+0000", 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0549999999999996e-06 2024-09-13T04:58:24.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718959+0000", 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5759999999999999e-06 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33283 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.167 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729017+0000", 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020416950999999999, 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027407330000000001, 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33283 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729017+0000", 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729017+0000", 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729017+0000", 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729020+0000", 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6089999999999999e-06 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729025+0000", 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6710000000000004e-06 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729030+0000", 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0590000000000002e-06 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.170 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33315 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731758+0000", 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017676587000000001, 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024738447, 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33315 2024-09-13T04:58:24.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731758+0000", 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731758+0000", 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731757+0000", 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731760+0000", 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5450000000000002e-06 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731766+0000", 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5740000000000001e-06 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731771+0000", 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9849999999999997e-06 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33347 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738368+0000", 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.011066258000000001, 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018189025000000001, 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33347 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738368+0000", 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738368+0000", 2024-09-13T04:58:24.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.734838+0000", 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.99647 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738375+0000", 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003537044 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738388+0000", 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2577e-05 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738410+0000", 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2050999999999999e-05 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33760 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750169+0000", 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9992652, 2024-09-13T04:58:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064605770000000003, 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33760 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750169+0000", 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750169+0000", 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750169+0000", 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750171+0000", 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9860000000000001e-06 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750176+0000", 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4870000000000002e-06 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750180+0000", 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1779999999999996e-06 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.180 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33027 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.710682+0000", 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.038752300000000003, 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04601529, 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33027 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710682+0000", 2024-09-13T04:58:24.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710682+0000", 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710681+0000", 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.710684+0000", 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4650000000000001e-06 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718244+0000", 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075596630000000003 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718258+0000", 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4681999999999999e-05 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33206 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718962+0000", 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030471846, 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037803306000000002, 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33206 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718962+0000", 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718962+0000", 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718962+0000", 2024-09-13T04:58:24.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718964+0000", 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0719999999999998e-06 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718970+0000", 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8170000000000004e-06 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718975+0000", 2024-09-13T04:58:24.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.605e-06 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33284 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729035+0000", 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020399739, 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027801184, 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33284 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729035+0000", 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729035+0000", 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729034+0000", 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729037+0000", 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.277e-06 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729043+0000", 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6220000000000003e-06 2024-09-13T04:58:24.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729048+0000", 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8269999999999997e-06 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33316 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731776+0000", 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017657900000000001, 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025125488000000001, 2024-09-13T04:58:24.190 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33316 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731776+0000", 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731776+0000", 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731776+0000", 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731778+0000", 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6180000000000002e-06 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731784+0000", 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1240000000000002e-06 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731798+0000", 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3061000000000001e-05 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33348 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.193 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738426+0000", 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.01100801, 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018539096000000001, 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33348 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738426+0000", 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738426+0000", 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738426+0000", 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738437+0000", 2024-09-13T04:58:24.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1746000000000001e-05 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738482+0000", 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4462999999999998e-05 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738491+0000", 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5659999999999996e-06 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33762 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750184+0000", 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9992499, 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068497239999999997, 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33762 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750184+0000", 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750184+0000", 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750184+0000", 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750186+0000", 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9410000000000001e-06 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750191+0000", 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4079999999999997e-06 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750195+0000", 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.755e-06 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.199 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33028 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718273+0000", 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.031161179000000001, 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038830366999999998, 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33028 2024-09-13T04:58:24.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718273+0000", 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718273+0000", 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718272+0000", 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718283+0000", 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1034e-05 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718293+0000", 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0801e-05 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718300+0000", 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2040000000000002e-06 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33207 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718984+0000", 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030450511, 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038182654000000003, 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33207 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718984+0000", 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718984+0000", 2024-09-13T04:58:24.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718983+0000", 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718986+0000", 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3190000000000002e-06 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718992+0000", 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718997+0000", 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7430000000000002e-06 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33253 3.f 3:f6d73b68:::rbd_data.112ab86ccaf9.0000000000000003:8 [read 2768896~16384 out=16384b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719635+0000", 2024-09-13T04:58:24.206 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.029799441999999999, 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037479855999999999, 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33253 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719635+0000", 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719635+0000", 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719634+0000", 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719637+0000", 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1519999999999999e-06 2024-09-13T04:58:24.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719643+0000", 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0589999999999999e-06 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719647+0000", 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5290000000000002e-06 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.755980+0000", 2024-09-13T04:58:24.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.036332296 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.756057+0000", 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7537e-05 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.757115+0000", 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001057611 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33285 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729053+0000", 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020381420000000001, 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028270956, 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33285 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729053+0000", 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729053+0000", 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729052+0000", 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729055+0000", 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5059999999999998e-06 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729061+0000", 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3099999999999997e-06 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729066+0000", 2024-09-13T04:58:24.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7960000000000003e-06 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33317 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731802+0000", 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.01763176, 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025583247999999999, 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33317 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731802+0000", 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731802+0000", 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731802+0000", 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731805+0000", 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.655e-06 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731812+0000", 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8850000000000001e-06 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731816+0000", 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8760000000000004e-06 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33349 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738498+0000", 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010936098, 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018941849, 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33349 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738498+0000", 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738498+0000", 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738498+0000", 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738501+0000", 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.89e-06 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738509+0000", 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7389999999999999e-06 2024-09-13T04:58:24.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738523+0000", 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.416e-05 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33763 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750199+0000", 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9992352, 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073113079999999999, 2024-09-13T04:58:24.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33763 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750199+0000", 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750199+0000", 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750199+0000", 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750201+0000", 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2749999999999998e-06 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750215+0000", 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4046000000000001e-05 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750220+0000", 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.758e-06 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33029 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718305+0000", 2024-09-13T04:58:24.223 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.031129567, 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039268336000000001, 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33029 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718305+0000", 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718305+0000", 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718304+0000", 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718307+0000", 2024-09-13T04:58:24.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.022e-06 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718314+0000", 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4439999999999996e-06 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718328+0000", 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4807000000000001e-05 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33208 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030433163999999999, 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038637523999999999, 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33208 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719003+0000", 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0389999999999999e-06 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719009+0000", 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7490000000000004e-06 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719013+0000", 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550000000000001e-06 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.229 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33286 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729070+0000", 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020363875, 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028641013, 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33286 2024-09-13T04:58:24.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729070+0000", 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729070+0000", 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729070+0000", 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729072+0000", 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0779999999999998e-06 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729078+0000", 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0850000000000002e-06 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729095+0000", 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6529e-05 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33318 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731821+0000", 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017613303, 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025956221000000002, 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33318 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731821+0000", 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731821+0000", 2024-09-13T04:58:24.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731821+0000", 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732218+0000", 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039765800000000001 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732231+0000", 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3284e-05 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732239+0000", 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3660000000000003e-06 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33350 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738529+0000", 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010905150000000001, 2024-09-13T04:58:24.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019308580999999998, 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33350 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738529+0000", 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738529+0000", 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738529+0000", 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738532+0000", 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5659999999999998e-06 2024-09-13T04:58:24.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738540+0000", 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9580000000000007e-06 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738546+0000", 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.947e-06 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.239 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33764 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750225+0000", 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9992094, 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0078464109999999993, 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33764 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750225+0000", 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750225+0000", 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750224+0000", 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750227+0000", 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.751e-06 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750233+0000", 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1229999999999997e-06 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750238+0000", 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4390000000000002e-06 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33030 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718334+0000", 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.031100506999999999, 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039807305000000001, 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33030 2024-09-13T04:58:24.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718334+0000", 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718334+0000", 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718333+0000", 2024-09-13T04:58:24.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718336+0000", 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718343+0000", 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1099999999999999e-06 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718347+0000", 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8330000000000001e-06 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030414493000000001, 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039211188000000001, 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33287 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729100+0000", 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020334189999999999, 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029173401000000002, 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.249 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33287 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729100+0000", 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729100+0000", 2024-09-13T04:58:24.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729100+0000", 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729102+0000", 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7080000000000002e-06 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729109+0000", 2024-09-13T04:58:24.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729114+0000", 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9019999999999998e-06 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33319 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.732246+0000", 2024-09-13T04:58:24.252 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017187866999999999, 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026092869000000001, 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33319 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732246+0000", 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732246+0000", 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732246+0000", 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732249+0000", 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9239999999999999e-06 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732256+0000", 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0960000000000003e-06 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732261+0000", 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3770000000000004e-06 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33351 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738551+0000", 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010883357999999999, 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019841877000000001, 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33351 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.256 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738551+0000", 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738551+0000", 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738551+0000", 2024-09-13T04:58:24.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738553+0000", 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6029999999999999e-06 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738560+0000", 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1219999999999997e-06 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738566+0000", 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4970000000000001e-06 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33765 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750241+0000", 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9991927, 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0082134370000000005, 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33765 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750241+0000", 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750241+0000", 2024-09-13T04:58:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750241+0000", 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750243+0000", 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750249+0000", 2024-09-13T04:58:24.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6980000000000003e-06 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750253+0000", 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0980000000000004e-06 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33031 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718352+0000", 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.031082347999999999, 2024-09-13T04:58:24.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040173312000000003, 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33031 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718352+0000", 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718352+0000", 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718352+0000", 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718359+0000", 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4089999999999997e-06 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718365+0000", 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2979999999999997e-06 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718370+0000", 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7899999999999999e-06 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.265 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030389101000000002, 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039543474000000002, 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.268 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33288 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729118+0000", 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020316245, 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029539180000000002, 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33288 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.269 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729118+0000", 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729118+0000", 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729118+0000", 2024-09-13T04:58:24.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729126+0000", 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9319999999999996e-06 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729133+0000", 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4170000000000003e-06 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729138+0000", 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2349999999999999e-06 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33320 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.732266+0000", 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017167970000000001, 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026453542, 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33320 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732266+0000", 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732266+0000", 2024-09-13T04:58:24.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732266+0000", 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732268+0000", 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3659999999999999e-06 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732274+0000", 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1580000000000002e-06 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732279+0000", 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7169999999999999e-06 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33352 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738571+0000", 2024-09-13T04:58:24.275 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010863401, 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020205417, 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33352 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738571+0000", 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738571+0000", 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738571+0000", 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738573+0000", 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5799999999999999e-06 2024-09-13T04:58:24.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738580+0000", 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6490000000000002e-06 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738592+0000", 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2289e-05 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33768 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750257+0000", 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.999177, 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085824850000000008, 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33768 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750257+0000", 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750257+0000", 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750257+0000", 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750266+0000", 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5690000000000005e-06 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750272+0000", 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0039999999999996e-06 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750276+0000", 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4959999999999998e-06 2024-09-13T04:58:24.281 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33032 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718374+0000", 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.031059870999999999, 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040552314999999999, 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33032 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718374+0000", 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718374+0000", 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718374+0000", 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718377+0000", 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5730000000000002e-06 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718382+0000", 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8490000000000002e-06 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718387+0000", 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5460000000000002e-06 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.03037165, 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039927685999999997, 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:58:24.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33289 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729143+0000", 2024-09-13T04:58:24.288 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.020291249000000001, 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029934632999999999, 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33289 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729143+0000", 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729143+0000", 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729143+0000", 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729145+0000", 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4609999999999998e-06 2024-09-13T04:58:24.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729534+0000", 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038895299999999999 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729543+0000", 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8519999999999993e-06 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33321 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.732283+0000", 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017151151999999999, 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026858882000000001, 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33321 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732283+0000", 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732283+0000", 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732283+0000", 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732285+0000", 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1009999999999999e-06 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732291+0000", 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.789e-06 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732295+0000", 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5859999999999998e-06 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.294 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33353 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738597+0000", 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010837253999999999, 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020613534999999999, 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33353 2024-09-13T04:58:24.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738597+0000", 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738597+0000", 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738596+0000", 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738600+0000", 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1779999999999999e-06 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738606+0000", 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3670000000000002e-06 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738611+0000", 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9239999999999998e-06 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33770 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750281+0000", 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9991536, 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0089951240000000002, 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.298 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33770 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750281+0000", 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750281+0000", 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750280+0000", 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750283+0000", 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750289+0000", 2024-09-13T04:58:24.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.772e-06 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750298+0000", 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0140000000000004e-06 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33033 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718391+0000", 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.031043382000000001, 2024-09-13T04:58:24.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040948761, 2024-09-13T04:58:24.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33033 2024-09-13T04:58:24.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.302 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718391+0000", 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718391+0000", 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718391+0000", 2024-09-13T04:58:24.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718393+0000", 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6359999999999998e-06 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718399+0000", 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9000000000000003e-06 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718404+0000", 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6380000000000003e-06 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030354401999999999, 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040329784, 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33290 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729551+0000", 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019883464999999999, 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029922093, 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33290 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729551+0000", 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729551+0000", 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729550+0000", 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729555+0000", 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7910000000000004e-06 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729565+0000", 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0382e-05 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729572+0000", 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7050000000000002e-06 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.312 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33322 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.732299+0000", 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.017134804999999999, 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027241451, 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33322 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732299+0000", 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732299+0000", 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732299+0000", 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732303+0000", 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0119999999999999e-06 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732310+0000", 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2869999999999999e-06 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732827+0000", 2024-09-13T04:58:24.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051653099999999998 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33354 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738616+0000", 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010818639, 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020991228000000001, 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33354 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738616+0000", 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738616+0000", 2024-09-13T04:58:24.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738615+0000", 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738631+0000", 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5392000000000001e-05 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738638+0000", 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2590000000000003e-06 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738643+0000", 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8520000000000003e-06 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33772 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750302+0000", 2024-09-13T04:58:24.319 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9991326, 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093768600000000007, 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33772 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750302+0000", 2024-09-13T04:58:24.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750302+0000", 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750301+0000", 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750304+0000", 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.317e-06 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750309+0000", 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6509999999999998e-06 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750314+0000", 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2270000000000004e-06 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.322 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33034 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718408+0000", 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.031026581000000001, 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041365163000000003, 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33034 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.323 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718408+0000", 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718408+0000", 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718407+0000", 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718410+0000", 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3130000000000001e-06 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718415+0000", 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7300000000000002e-06 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718419+0000", 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0149999999999997e-06 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030331648999999999, 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040752380999999997, 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:58:24.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33291 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729578+0000", 2024-09-13T04:58:24.329 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.01985636, 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030339588000000001, 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33291 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729578+0000", 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729578+0000", 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729577+0000", 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729580+0000", 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.756e-06 2024-09-13T04:58:24.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729596+0000", 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5429000000000001e-05 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729602+0000", 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33323 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.732836+0000", 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.016598124999999998, 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027149408, 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33323 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732836+0000", 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732836+0000", 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732835+0000", 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732840+0000", 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4410000000000003e-06 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732850+0000", 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.023e-05 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732856+0000", 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1380000000000004e-06 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.335 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33355 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738648+0000", 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010786425000000001, 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021392442000000001, 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33355 2024-09-13T04:58:24.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738648+0000", 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738648+0000", 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738647+0000", 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738650+0000", 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5560000000000002e-06 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738656+0000", 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5470000000000004e-06 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738660+0000", 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6140000000000002e-06 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33774 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750317+0000", 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9991169, 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097593100000000002, 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33774 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750317+0000", 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750317+0000", 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750317+0000", 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750319+0000", 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9860000000000001e-06 2024-09-13T04:58:24.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750325+0000", 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750330+0000", 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3630000000000004e-06 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33036 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718424+0000", 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.031010527999999999, 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041692360999999997, 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33036 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718424+0000", 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718424+0000", 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718423+0000", 2024-09-13T04:58:24.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718460+0000", 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6356000000000002e-05 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718468+0000", 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6999999999999997e-06 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718480+0000", 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1094000000000001e-05 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030314699000000001, 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041045166000000001, 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33292 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729608+0000", 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019826417999999998, 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030602404999999999, 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33292 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729608+0000", 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729608+0000", 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729607+0000", 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729610+0000", 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.88e-06 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729618+0000", 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3089999999999998e-06 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729623+0000", 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2870000000000004e-06 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.352 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33324 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.732862+0000", 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.016572630000000001, 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027410308000000001, 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33324 2024-09-13T04:58:24.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.354 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732862+0000", 2024-09-13T04:58:24.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732862+0000", 2024-09-13T04:58:24.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732861+0000", 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732872+0000", 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1088e-05 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732881+0000", 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7949999999999996e-06 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732886+0000", 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519999999999997e-06 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33356 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.357 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738665+0000", 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010769604, 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021674054000000002, 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33356 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738665+0000", 2024-09-13T04:58:24.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738665+0000", 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738664+0000", 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738667+0000", 2024-09-13T04:58:24.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738673+0000", 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0820000000000004e-06 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738677+0000", 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4680000000000003e-06 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33776 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750333+0000", 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9991007, 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010068453999999999, 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33776 2024-09-13T04:58:24.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750333+0000", 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750333+0000", 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750333+0000", 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750339+0000", 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1480000000000003e-06 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750345+0000", 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6180000000000003e-06 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750355+0000", 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6909999999999992e-06 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33038 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718485+0000", 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030949258, 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041990723000000001, 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33038 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718485+0000", 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718485+0000", 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718485+0000", 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718487+0000", 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6110000000000001e-06 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718494+0000", 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4269999999999996e-06 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718498+0000", 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7620000000000003e-06 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:58:24.367 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030293033, 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041398796000000002, 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:58:24.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33261 3.f 3:f6d73b68:::rbd_data.112ab86ccaf9.0000000000000003:8 [read 3031040~16384 out=16384b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.724898+0000", 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.024535953999999999, 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035706935000000002, 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33261 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.371 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.724898+0000", 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.724898+0000", 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.724898+0000", 2024-09-13T04:58:24.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.724901+0000", 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4340000000000001e-06 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.724908+0000", 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1459999999999999e-06 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.724914+0000", 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3809999999999998e-06 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.760108+0000", 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035194667999999998 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.760189+0000", 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0201e-05 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.760605+0000", 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041672699999999998 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33293 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729628+0000", 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019806471999999999, 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031063016999999998, 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33293 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729628+0000", 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729628+0000", 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729627+0000", 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729630+0000", 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3819999999999998e-06 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729636+0000", 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729641+0000", 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8620000000000002e-06 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33325 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.378 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.732895+0000", 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.016539732000000001, 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027862327999999999, 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33325 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732895+0000", 2024-09-13T04:58:24.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732895+0000", 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732894+0000", 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732897+0000", 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.841e-06 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732903+0000", 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6259999999999997e-06 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732908+0000", 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0039999999999999e-06 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33357 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738681+0000", 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010752952, 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022136266000000002, 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33357 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738681+0000", 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738681+0000", 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738681+0000", 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738683+0000", 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2129999999999998e-06 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738690+0000", 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3940000000000001e-06 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738695+0000", 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9599999999999999e-06 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33778 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750359+0000", 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9990754, 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010521674999999999, 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33778 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750359+0000", 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750359+0000", 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750359+0000", 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750361+0000", 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750368+0000", 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8390000000000001e-06 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750372+0000", 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2509999999999996e-06 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33040 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718503+0000", 2024-09-13T04:58:24.388 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030931681999999999, 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042446428000000001, 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33040 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718503+0000", 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718503+0000", 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718502+0000", 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718504+0000", 2024-09-13T04:58:24.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1210000000000001e-06 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718510+0000", 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.631e-06 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718515+0000", 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5789999999999997e-06 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030276798000000001, 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041858332999999998, 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:58:24.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33294 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729650+0000", 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019784415, 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031429413000000003, 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33294 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729650+0000", 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729650+0000", 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729649+0000", 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729652+0000", 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.695e-06 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729659+0000", 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6120000000000004e-06 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729663+0000", 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7570000000000004e-06 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33326 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.732913+0000", 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.016521184000000001, 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028237964000000001, 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33326 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732913+0000", 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732913+0000", 2024-09-13T04:58:24.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732913+0000", 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.732915+0000", 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3140000000000002e-06 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733352+0000", 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00043703699999999998 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733360+0000", 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6020000000000002e-06 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33358 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738699+0000", 2024-09-13T04:58:24.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010735473000000001, 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022515080999999999, 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33358 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738699+0000", 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738699+0000", 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738698+0000", 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738701+0000", 2024-09-13T04:58:24.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2259999999999999e-06 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738707+0000", 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1059999999999996e-06 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738720+0000", 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2787e-05 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33780 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750376+0000", 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9990582, 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010905653, 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33780 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750376+0000", 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750376+0000", 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750376+0000", 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750377+0000", 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7969999999999999e-06 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750383+0000", 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0919999999999997e-06 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750387+0000", 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0219999999999998e-06 2024-09-13T04:58:24.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33042 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.718519+0000", 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030915584999999999, 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042828307000000003, 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33042 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718519+0000", 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718519+0000", 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718518+0000", 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718521+0000", 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3300000000000001e-06 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718526+0000", 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7479999999999999e-06 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.718531+0000", 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3680000000000004e-06 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.030260485, 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042236277000000003, 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33295 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729668+0000", 2024-09-13T04:58:24.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.019765885, 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031807213000000001, 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33295 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729668+0000", 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729668+0000", 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729668+0000", 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729671+0000", 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.089e-06 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729677+0000", 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.285e-06 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729692+0000", 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5167000000000001e-05 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33327 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.733368+0000", 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.016066081999999999, 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028168364000000001, 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33327 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733368+0000", 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733368+0000", 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733367+0000", 2024-09-13T04:58:24.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733372+0000", 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5639999999999998e-06 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733380+0000", 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0989999999999999e-06 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.733385+0000", 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.215e-06 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33359 3.f 3.16dceb6f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.738725+0000", 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.010708871999999999, 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022877172000000001, 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33359 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738725+0000", 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738725+0000", 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738725+0000", 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738728+0000", 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7460000000000001e-06 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738734+0000", 2024-09-13T04:58:24.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1539999999999999e-06 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.738738+0000", 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7310000000000001e-06 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33782 3.3 3.342e263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.750390+0000", 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9990439, 2024-09-13T04:58:24.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011275721000000001, 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33782 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750390+0000", 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750390+0000", 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750390+0000", 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750392+0000", 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.742e-06 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750397+0000", 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1930000000000001e-06 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.750401+0000", 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7290000000000002e-06 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:24.427 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 195 2024-09-13T04:58:24.428 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:24.428 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:24.428 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:24.430 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:24.431 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-13T04:58:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:985 write 0x2c34648 thru 0x2c364d6 (0x1e8f bytes) 2024-09-13T04:58:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:987 read 0xaaea728 thru 0xaaecc43 (0x251c bytes) 2024-09-13T04:58:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:989 writesame 0x425cf08 thru 0x4264811 data_size 0x1(0x790a bytes) 2024-09-13T04:58:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:991 read 0x2069ecc thru 0x206e6de (0x4813 bytes) 2024-09-13T04:58:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:992 punch from 0xa0a9f36 to 0xa0b3eb9, (0x9f83 bytes) 2024-09-13T04:58:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:993 trunc from 0xd01af84 to 0x35ac9b 2024-09-13T04:58:24.433 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:58:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:994 read 0x486f3 thru 0x492f8 (0xc06 bytes) 2024-09-13T04:58:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:995 write 0x923d7d2 thru 0x924a831 (0xd060 bytes) 2024-09-13T04:58:24.434 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:24.428+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:24.435 INFO:teuthology.orchestra.run.smithi001.stdout:996 write 0x90fd5c7 thru 0x90fe6bc (0x10f6 bytes) 2024-09-13T04:58:24.444 INFO:teuthology.orchestra.run.smithi001.stdout:998 trunc from 0x924a832 to 0xa82fe64 2024-09-13T04:58:24.458 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:24.458 INFO:teuthology.orchestra.run.smithi001.stdout:1000 read 0x9034b48 thru 0x9037481 (0x293a bytes) 2024-09-13T04:58:24.459 INFO:teuthology.orchestra.run.smithi001.stdout:1001 punch from 0x8f4f9d7 to 0x8f59efd, (0xa526 bytes) 2024-09-13T04:58:24.461 INFO:teuthology.orchestra.run.smithi001.stdout:1002 trunc from 0xa82fe64 to 0x9118c7a 2024-09-13T04:58:24.484 INFO:teuthology.orchestra.run.smithi001.stdout:1003 write 0x23023aa thru 0x230ecb8 (0xc90f bytes) 2024-09-13T04:58:24.518 INFO:teuthology.orchestra.run.smithi001.stdout:1004 write 0xcd9f59c thru 0xcdac023 (0xca88 bytes) 2024-09-13T04:58:24.525 INFO:teuthology.orchestra.run.smithi001.stdout:1005 write 0xcb3d505 thru 0xcb3e36e (0xe6a bytes) 2024-09-13T04:58:24.531 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:24.528+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:24.531 INFO:teuthology.orchestra.run.smithi001.stdout:1006 write 0x66e1056 thru 0x66e18b1 (0x85c bytes) 2024-09-13T04:58:24.538 INFO:teuthology.orchestra.run.smithi001.stdout:1007 read 0x2648f2c thru 0x26505e0 (0x76b5 bytes) 2024-09-13T04:58:24.539 INFO:teuthology.orchestra.run.smithi001.stdout:1008 punch from 0x1bb7d73 to 0x1bbb952, (0x3bdf bytes) 2024-09-13T04:58:24.554 INFO:teuthology.orchestra.run.smithi001.stdout:1009 write 0xe96f8dc thru 0xe97a3fa (0xab1f bytes) 2024-09-13T04:58:24.556 INFO:teuthology.orchestra.run.smithi001.stdout:1010 read 0x6d3bab2 thru 0x6d3dbce (0x211d bytes) 2024-09-13T04:58:24.557 INFO:teuthology.orchestra.run.smithi001.stdout:1011 read 0x7945356 thru 0x794b0dc (0x5d87 bytes) 2024-09-13T04:58:24.557 INFO:teuthology.orchestra.run.smithi001.stdout:1012 write 0xa8de8be thru 0xa8ed34a (0xea8d bytes) 2024-09-13T04:58:24.558 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:24.558 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:24.558 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:24.570 INFO:teuthology.orchestra.run.smithi001.stdout:1013 writesame 0xaca3714 thru 0xacae909 data_size 0x9(0xb1f6 bytes) 2024-09-13T04:58:24.572 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:24.579 INFO:teuthology.orchestra.run.smithi001.stdout:1014 writesame 0x663a8ec thru 0x663c700 data_size 0x11(0x1e15 bytes) 2024-09-13T04:58:24.581 INFO:teuthology.orchestra.run.smithi001.stdout:1015 read 0xc5f510c thru 0xc5f5247 (0x13c bytes) 2024-09-13T04:58:24.583 INFO:teuthology.orchestra.run.smithi001.stdout:1016 trunc from 0xe97a3fb to 0xd63a464 2024-09-13T04:58:24.596 INFO:teuthology.orchestra.run.smithi001.stdout:1017 write 0x1805c96 thru 0x180beb0 (0x621b bytes) 2024-09-13T04:58:24.613 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T04:58:24.618 INFO:teuthology.orchestra.run.smithi001.stdout:1018 trunc from 0xd63a464 to 0x2bd5100 2024-09-13T04:58:24.642 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:24.642 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:24.640+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:24.646 INFO:teuthology.orchestra.run.smithi001.stdout:1022 trunc from 0x2bd5100 to 0x617c32b 2024-09-13T04:58:24.658 INFO:teuthology.orchestra.run.smithi001.stdout:1023 trunc from 0x617c32b to 0x4ed78dd 2024-09-13T04:58:24.695 INFO:teuthology.orchestra.run.smithi001.stdout:1025 writesame 0xcb88aee thru 0xcb967bc data_size 0x1(0xdccf bytes) 2024-09-13T04:58:24.695 INFO:teuthology.orchestra.run.smithi001.stdout:1027 read 0x6af2631 thru 0x6af8fc2 (0x6992 bytes) 2024-09-13T04:58:24.696 INFO:teuthology.orchestra.run.smithi001.stdout:1028 trunc from 0xcb967bd to 0xbc19e3e 2024-09-13T04:58:24.697 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:24.697 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33208 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97783401999999997, 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053460584999999998, 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33208 2024-09-13T04:58:24.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:58:24.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719003+0000", 2024-09-13T04:58:24.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0389999999999999e-06 2024-09-13T04:58:24.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719009+0000", 2024-09-13T04:58:24.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7490000000000004e-06 2024-09-13T04:58:24.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719013+0000", 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550000000000001e-06 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772013+0000", 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053000688999999997 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772063+0000", 2024-09-13T04:58:24.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9774000000000002e-05 2024-09-13T04:58:24.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772462+0000", 2024-09-13T04:58:24.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039860600000000003 2024-09-13T04:58:24.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.704 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.708 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.708 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:58:24.708 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.977815349, 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053636234999999997, 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772488+0000", 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053447254999999999 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772562+0000", 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4276999999999994e-05 2024-09-13T04:58:24.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772656+0000", 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3993999999999995e-05 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5898240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97778995700000004, 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054161726, 2024-09-13T04:58:24.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:58:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:58:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:58:24.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772681+0000", 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053622832000000002 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772751+0000", 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0166999999999994e-05 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773207+0000", 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045547300000000002 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5931008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97777250599999999, 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054348847999999998, 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.721 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:58:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773235+0000", 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054159997000000001 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773316+0000", 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0727000000000001e-05 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773411+0000", 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5266999999999994e-05 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97775525799999996, 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054777692000000003, 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:58:24.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773440+0000", 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054341649999999998 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773514+0000", 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4501999999999999e-05 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773858+0000", 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034316700000000002 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:58:24.729 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97773250499999997, 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054958912999999998, 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:58:24.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773886+0000", 2024-09-13T04:58:24.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054770389000000003 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773964+0000", 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774062+0000", 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7059999999999996e-05 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97771555499999996, 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055382103000000002, 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:58:24.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774088+0000", 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054950812000000002 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774163+0000", 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5084000000000001e-05 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774502+0000", 2024-09-13T04:58:24.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033872500000000001 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97769388899999998, 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055600325999999999, 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:58:24.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774532+0000", 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055378563999999998 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774633+0000", 2024-09-13T04:58:24.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100519 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774742+0000", 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010891900000000001 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6094848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97767765399999995, 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056713126000000003, 2024-09-13T04:58:24.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774769+0000", 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055599061999999998 2024-09-13T04:58:24.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774843+0000", 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3801999999999996e-05 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775871+0000", 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00102797 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97766134100000002, 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056923247000000003, 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775941+0000", 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056755141000000002 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775996+0000", 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4580999999999998e-05 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776097+0000", 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101456 2024-09-13T04:58:24.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97764534000000003, 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057407979999999997, 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:58:24.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:58:24.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776138+0000", 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056931362999999999 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776293+0000", 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015495600000000001 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776598+0000", 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030449200000000001 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33298 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96710149700000003, 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053286770999999997, 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33298 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729733+0000", 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729735+0000", 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.875e-06 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729741+0000", 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.417e-06 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729754+0000", 2024-09-13T04:58:24.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3811999999999999e-05 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.782831+0000", 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053076726999999997 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.782926+0000", 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5177e-05 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783020+0000", 2024-09-13T04:58:24.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4040000000000001e-05 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96705655800000001, 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053642026000000002, 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783061+0000", 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053263004000000003 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783140+0000", 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9290999999999997e-05 2024-09-13T04:58:24.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783421+0000", 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028007100000000001 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96703207400000002, 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053832005000000002, 2024-09-13T04:58:24.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783450+0000", 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053635404999999997 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783532+0000", 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1310000000000006e-05 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783635+0000", 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010324599999999999 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.768 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96701610599999999, 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05399698, 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783666+0000", 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053834224999999999 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783745+0000", 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8542999999999997e-05 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783816+0000", 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279999999999995e-05 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.772 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96698283500000004, 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054125706000000003, 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.773 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:58:24.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:58:24.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783842+0000", 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053971225999999997 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783915+0000", 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2657999999999995e-05 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783978+0000", 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3157000000000005e-05 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33307 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96585427000000001, 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053424891000000002, 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33307 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:58:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730980+0000", 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730983+0000", 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5359999999999999e-06 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730989+0000", 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730994+0000", 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8040000000000001e-06 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784016+0000", 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053022133999999999 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784195+0000", 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017906399999999999 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784406+0000", 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021028099999999999 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.781 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96581930299999996, 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053864002000000001, 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:58:24.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784436+0000", 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053408069000000002 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784643+0000", 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000207171 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784880+0000", 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023680700000000001 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.785 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96578687500000004, 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054149968, 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.786 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:58:24.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:58:24.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784958+0000", 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053896985000000001 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785046+0000", 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8321999999999999e-05 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785198+0000", 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015185 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96512524200000005, 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053689033999999997, 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:24.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:58:24.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785224+0000", 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053491874000000002 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785304+0000", 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9825000000000002e-05 2024-09-13T04:58:24.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785399+0000", 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4837e-05 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:24.794 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:24.795 INFO:teuthology.orchestra.run.smithi001.stdout:1029 trunc from 0xbc19e3e to 0x9ac75d2 2024-09-13T04:58:24.796 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:24.797 INFO:teuthology.orchestra.run.smithi001.stdout:1030 punch from 0x39b0c3c to 0x39baf53, (0xa317 bytes) 2024-09-13T04:58:24.797 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:24.797 INFO:teuthology.orchestra.run.smithi001.stdout:1033 read 0x132a400 thru 0x1339a03 (0xf604 bytes) 2024-09-13T04:58:24.797 INFO:teuthology.orchestra.run.smithi001.stdout:1034 trunc from 0x9ac75d2 to 0xe9db20d 2024-09-13T04:58:24.797 INFO:teuthology.orchestra.run.smithi001.stdout:1035 write 0x67e8097 thru 0x67e87bd (0x727 bytes) 2024-09-13T04:58:24.798 INFO:teuthology.orchestra.run.smithi001.stdout:1036 write 0xa3ae83f thru 0xa3be3be (0xfb80 bytes) 2024-09-13T04:58:24.798 INFO:teuthology.orchestra.run.smithi001.stdout:1037 write 0x8e51a98 thru 0x8e527dc (0xd45 bytes) 2024-09-13T04:58:24.798 INFO:teuthology.orchestra.run.smithi001.stdout:1039 writesame 0x1687995 thru 0x1694136 data_size 0x2e(0xc7a2 bytes) 2024-09-13T04:58:24.798 INFO:teuthology.orchestra.run.smithi001.stdout:1041 read 0x10d8f3c thru 0x10e38bd (0xa982 bytes) 2024-09-13T04:58:24.798 INFO:teuthology.orchestra.run.smithi001.stdout:1042 punch from 0xabfb5d0 to 0xac09b14, (0xe544 bytes) 2024-09-13T04:58:24.798 INFO:teuthology.orchestra.run.smithi001.stdout:1043 write 0xc6a84c7 thru 0xc6af2d6 (0x6e10 bytes) 2024-09-13T04:58:24.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T04:58:24.799 INFO:teuthology.orchestra.run.smithi001.stdout:1044 punch from 0x4c0c2bf to 0x4c1bced, (0xfa2e bytes) 2024-09-13T04:58:24.799 INFO:teuthology.orchestra.run.smithi001.stdout:1045 trunc from 0xe9db20d to 0xa4a3d26 2024-09-13T04:58:24.799 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:24.792+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:24.810 INFO:teuthology.orchestra.run.smithi001.stdout:1046 write 0xa2f1d39 thru 0xa2fbc16 (0x9ede bytes) 2024-09-13T04:58:24.817 INFO:teuthology.orchestra.run.smithi001.stdout:1047 punch from 0x1c625ac to 0x1c66fce, (0x4a22 bytes) 2024-09-13T04:58:24.832 INFO:teuthology.orchestra.run.smithi001.stdout:1048 write 0xcef1386 thru 0xcefe691 (0xd30c bytes) 2024-09-13T04:58:24.841 INFO:teuthology.orchestra.run.smithi001.stdout:1049 write 0x83655c2 thru 0x836d50f (0x7f4e bytes) 2024-09-13T04:58:24.847 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:24.847 INFO:teuthology.orchestra.run.smithi001.stdout:1051 punch from 0xf875be to 0xf8c7dc, (0x521e bytes) 2024-09-13T04:58:24.847 INFO:teuthology.orchestra.run.smithi001.stdout:1052 write 0xa1ee40d thru 0xa1f2c6d (0x4861 bytes) 2024-09-13T04:58:24.849 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:24.849 INFO:teuthology.orchestra.run.smithi001.stdout:1055 write 0x87f6582 thru 0x87ff3d4 (0x8e53 bytes) 2024-09-13T04:58:24.860 INFO:teuthology.orchestra.run.smithi001.stdout:1056 writesame 0x274ec93 thru 0x2759e38 data_size 0x1(0xb1a6 bytes) 2024-09-13T04:58:24.896 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:24.892+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:24.911 INFO:teuthology.orchestra.run.smithi001.stdout:1057 trunc from 0xcefe692 to 0xadad5ac 2024-09-13T04:58:24.923 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:24.924 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:24.924 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:24.924 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:24.938 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:24.954 INFO:teuthology.orchestra.run.smithi001.stdout:1058 writesame 0xbf0591f thru 0xbf09534 data_size 0x1(0x3c16 bytes) 2024-09-13T04:58:24.954 INFO:teuthology.orchestra.run.smithi001.stdout:1059 trunc from 0xbf09535 to 0xc58ad15 2024-09-13T04:58:24.965 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-13T04:58:24.966 INFO:teuthology.orchestra.run.smithi001.stdout:1060 writesame 0x43448b1 thru 0x434af53 data_size 0x5(0x66a3 bytes) 2024-09-13T04:58:24.975 INFO:teuthology.orchestra.run.smithi001.stdout:1061 writesame 0x8a0c281 thru 0x8a19f1a data_size 0x97(0xdc9a bytes) 2024-09-13T04:58:24.984 INFO:teuthology.orchestra.run.smithi001.stdout:1062 writesame 0x310547 thru 0x31a3df data_size 0x1(0x9e99 bytes) 2024-09-13T04:58:24.997 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:24.996+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:25.008 INFO:teuthology.orchestra.run.smithi001.stdout:1064 read 0x5e518a9 thru 0x5e5f5d9 (0xdd31 bytes) 2024-09-13T04:58:25.008 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.008 INFO:teuthology.orchestra.run.smithi001.stdout:1066 writesame 0x9f0ba8f thru 0x9f10be8 data_size 0x1(0x515a bytes) 2024-09-13T04:58:25.025 INFO:teuthology.orchestra.run.smithi001.stdout:1067 read 0x210a5da thru 0x2117d89 (0xd7b0 bytes) 2024-09-13T04:58:25.026 INFO:teuthology.orchestra.run.smithi001.stdout:1068 writesame 0xad0eb54 thru 0xad17a9c data_size 0x1(0x8f49 bytes) 2024-09-13T04:58:25.053 INFO:teuthology.orchestra.run.smithi001.stdout:1069 punch from 0x3516469 to 0x351b76c, (0x5303 bytes) 2024-09-13T04:58:25.053 INFO:teuthology.orchestra.run.smithi001.stdout:1070 punch from 0x6beeb20 to 0x6bfcef4, (0xe3d4 bytes) 2024-09-13T04:58:25.054 INFO:teuthology.orchestra.run.smithi001.stdout:1071 read 0x677546e thru 0x6779b75 (0x4708 bytes) 2024-09-13T04:58:25.054 INFO:teuthology.orchestra.run.smithi001.stdout:1072 writesame 0x549fc51 thru 0x54a2d7a data_size 0x3e(0x312a bytes) 2024-09-13T04:58:25.062 INFO:teuthology.orchestra.run.smithi001.stdout:1074 read 0x6513ee2 thru 0x65143bd (0x4dc bytes) 2024-09-13T04:58:25.077 INFO:teuthology.orchestra.run.smithi001.stdout:1075 writesame 0xd16177d thru 0xd16cd0f data_size 0x2b(0xb593 bytes) 2024-09-13T04:58:25.077 INFO:teuthology.orchestra.run.smithi001.stdout:1077 write 0x2fa4d27 thru 0x2faf00d (0xa2e7 bytes) 2024-09-13T04:58:25.079 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:25.079 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:25.079 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:25.079 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:25.079 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:25.086 INFO:teuthology.orchestra.run.smithi001.stdout:1081 write 0x89ca59c thru 0x89d1fc8 (0x7a2d bytes) 2024-09-13T04:58:25.094 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:25.096 INFO:teuthology.orchestra.run.smithi001.stdout:1082 read 0x28f598a thru 0x28f6db0 (0x1427 bytes) 2024-09-13T04:58:25.097 INFO:teuthology.orchestra.run.smithi001.stdout:1083 write 0x33b917b thru 0x33c8d88 (0xfc0e bytes) 2024-09-13T04:58:25.098 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:25.096+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:25.105 INFO:teuthology.orchestra.run.smithi001.stdout:1084 writesame 0x59c5619 thru 0x59c7fec data_size 0x2(0x29d4 bytes) 2024-09-13T04:58:25.114 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.114 INFO:teuthology.orchestra.run.smithi001.stdout:1088 writesame 0x17d9745 thru 0x17e5eb5 data_size 0x5d(0xc771 bytes) 2024-09-13T04:58:25.116 INFO:teuthology.orchestra.run.smithi001.stdout:1089 writesame 0x786521b thru 0x786a956 data_size 0x6(0x573c bytes) 2024-09-13T04:58:25.124 INFO:teuthology.orchestra.run.smithi001.stdout:1090 punch from 0x9d3289d to 0x9d3c3de, (0x9b41 bytes) 2024-09-13T04:58:25.125 INFO:teuthology.orchestra.run.smithi001.stdout:1091 read 0x8a7f547 thru 0x8a8dfad (0xea67 bytes) 2024-09-13T04:58:25.137 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-13T04:58:25.138 INFO:teuthology.orchestra.run.smithi001.stdout:1092 writesame 0xe307e1b thru 0xe3168f9 data_size 0x1(0xeadf bytes) 2024-09-13T04:58:25.138 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.138 INFO:teuthology.orchestra.run.smithi001.stdout:1094 write 0xd5f09e6 thru 0xd5f38ca (0x2ee5 bytes) 2024-09-13T04:58:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:1096 read 0xa44ba7d thru 0xa4547db (0x8d5f bytes) 2024-09-13T04:58:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:1098 write 0xabceff7 thru 0xabd1fce (0x2fd8 bytes) 2024-09-13T04:58:25.152 INFO:teuthology.orchestra.run.smithi001.stdout:1099 read 0x4899975 thru 0x48a37d3 (0x9e5f bytes) 2024-09-13T04:58:25.153 INFO:teuthology.orchestra.run.smithi001.stdout:1100 write 0x795ebca thru 0x796d453 (0xe88a bytes) 2024-09-13T04:58:25.164 INFO:teuthology.orchestra.run.smithi001.stdout:1101 writesame 0xd70068a thru 0xd70b0a8 data_size 0x9(0xaa1f bytes) 2024-09-13T04:58:25.171 INFO:teuthology.orchestra.run.smithi001.stdout:1102 writesame 0x677e405 thru 0x67898f4 data_size 0x78(0xb4f0 bytes) 2024-09-13T04:58:25.175 INFO:teuthology.orchestra.run.smithi001.stdout:1104 read 0x72d1c37 thru 0x72d3af2 (0x1ebc bytes) 2024-09-13T04:58:25.175 INFO:teuthology.orchestra.run.smithi001.stdout:1105 write 0x2d19bc2 thru 0x2d1a524 (0x963 bytes) 2024-09-13T04:58:25.182 INFO:teuthology.orchestra.run.smithi001.stdout:1108 write 0x131d2bd thru 0x1320a7f (0x37c3 bytes) 2024-09-13T04:58:25.189 INFO:teuthology.orchestra.run.smithi001.stdout:1109 punch from 0x3da1104 to 0x3dae4a7, (0xd3a3 bytes) 2024-09-13T04:58:25.189 INFO:teuthology.orchestra.run.smithi001.stdout:1110 read 0x6f49154 thru 0x6f4c426 (0x32d3 bytes) 2024-09-13T04:58:25.189 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.189 INFO:teuthology.orchestra.run.smithi001.stdout:1113 write 0x1e4ccd2 thru 0x1e4d257 (0x586 bytes) 2024-09-13T04:58:25.190 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.190 INFO:teuthology.orchestra.run.smithi001.stdout:1116 write 0x83bf85d thru 0x83c2873 (0x3017 bytes) 2024-09-13T04:58:25.192 INFO:teuthology.orchestra.run.smithi001.stdout:1117 read 0x2e9b994 thru 0x2e9e523 (0x2b90 bytes) 2024-09-13T04:58:25.192 INFO:teuthology.orchestra.run.smithi001.stdout:1119 write 0xb44513 thru 0xb51e2b (0xd919 bytes) 2024-09-13T04:58:25.198 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:25.196+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:25.200 INFO:teuthology.orchestra.run.smithi001.stdout:1121 read 0x328782f thru 0x3295769 (0xdf3b bytes) 2024-09-13T04:58:25.209 INFO:teuthology.orchestra.run.smithi001.stdout:1122 trunc from 0xe3168fa to 0x63a8dfe 2024-09-13T04:58:25.224 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:25.224 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:25.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:25.224 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16592 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1876469069999995, 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055701260000000002, 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16592 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036513+0000", 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5299999999999999e-06 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036521+0000", 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6699999999999994e-06 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036525+0000", 2024-09-13T04:58:25.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3610000000000003e-06 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092051+0000", 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05552613 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092163+0000", 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111494 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092212+0000", 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9432999999999999e-05 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16593 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1876283870000002, 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055799934000000002, 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16593 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036532+0000", 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3810000000000002e-06 2024-09-13T04:58:25.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036539+0000", 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2139999999999999e-06 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036544+0000", 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1909999999999999e-06 2024-09-13T04:58:25.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092237+0000", 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692609999999997 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092290+0000", 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3554000000000003e-05 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092329+0000", 2024-09-13T04:58:25.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9335999999999997e-05 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16594 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1876094130000006, 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055959726000000001, 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16594 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036550+0000", 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036557+0000", 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999999e-06 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036562+0000", 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092364+0000", 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801515000000003 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092441+0000", 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7044000000000003e-05 2024-09-13T04:58:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092508+0000", 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7617e-05 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16595 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1872577050000004, 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055757809999999998, 2024-09-13T04:58:25.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16595 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036566+0000", 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996657 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036905+0000", 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033883999999999997 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036915+0000", 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0164e-05 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036930+0000", 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5245999999999999e-05 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092559+0000", 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055628628999999999 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092615+0000", 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6684000000000001e-05 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092658+0000", 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2559999999999999e-05 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.242 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16596 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3215360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.18702006, 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696469999999998, 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16596 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:25.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036936+0000", 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997983 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037142+0000", 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020614900000000001 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037152+0000", 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6739999999999992e-06 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037159+0000", 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092692+0000", 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055533569999999997 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092768+0000", 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5220000000000001e-05 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092834+0000", 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6674999999999994e-05 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.246 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16597 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1868671989999999, 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055678257000000002, 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16597 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037164+0000", 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998736 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037300+0000", 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000135456 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037310+0000", 2024-09-13T04:58:25.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0366e-05 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037316+0000", 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4910000000000002e-06 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092888+0000", 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055571872000000001 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092929+0000", 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0667999999999998e-05 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092969+0000", 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9913000000000002e-05 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16603 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1858388249999994, 2024-09-13T04:58:25.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055630449999999998, 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16603 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038321+0000", 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1179999999999999e-06 2024-09-13T04:58:25.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038326+0000", 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6189999999999999e-06 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038331+0000", 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2429999999999999e-06 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093860+0000", 2024-09-13T04:58:25.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055528871 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093900+0000", 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0742999999999999e-05 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093949+0000", 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16604 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1858231959999994, 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055794816999999997, 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16604 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038334+0000", 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038336+0000", 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.046e-06 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038342+0000", 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038346+0000", 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1799999999999998e-06 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093983+0000", 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055636906999999999 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094059+0000", 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6075999999999995e-05 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094129+0000", 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0337000000000001e-05 2024-09-13T04:58:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16606 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1853143839999998, 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055591046999999998, 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16606 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038845+0000", 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4329999999999998e-06 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038852+0000", 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1240000000000002e-06 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038857+0000", 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9620000000000001e-06 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094307+0000", 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055450714999999998 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094392+0000", 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4586000000000006e-05 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094435+0000", 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2697000000000001e-05 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16607 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1852972459999993, 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696915999999999, 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.264 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16607 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:25.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038860+0000", 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038862+0000", 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1859999999999999e-06 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038868+0000", 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8590000000000001e-06 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038873+0000", 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6310000000000002e-06 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094459+0000", 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055585561999999998 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094519+0000", 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0161999999999998e-05 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094558+0000", 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8831999999999997e-05 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16608 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1852809870000005, 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801370000000003, 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16608 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038879+0000", 2024-09-13T04:58:25.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.21e-06 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038885+0000", 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8100000000000003e-06 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038889+0000", 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2760000000000002e-06 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094581+0000", 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692236999999999 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094639+0000", 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7935e-05 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094678+0000", 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9267999999999997e-05 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16609 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1850511109999999, 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055693234000000001, 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16609 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.273 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038892+0000", 2024-09-13T04:58:25.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997854 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039112+0000", 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000219402 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039125+0000", 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039135+0000", 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8549999999999997e-06 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094702+0000", 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055567083000000003 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094761+0000", 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9425000000000001e-05 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094800+0000", 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8754999999999997e-05 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16610 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1849660039999996, 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0557396, 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16610 2024-09-13T04:58:25.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.278 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:25.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039158+0000", 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999666 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039196+0000", 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7551e-05 2024-09-13T04:58:25.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039212+0000", 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6569999999999999e-05 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039219+0000", 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5080000000000002e-06 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094824+0000", 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055604690999999998 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094890+0000", 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6204999999999994e-05 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094931+0000", 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1675000000000003e-05 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16636 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1812571950000006, 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055573821000000002, 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16636 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042900+0000", 2024-09-13T04:58:25.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042902+0000", 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0590000000000001e-06 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042908+0000", 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5620000000000003e-06 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042912+0000", 2024-09-13T04:58:25.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.437e-06 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098321+0000", 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055408959000000001 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098401+0000", 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0122000000000003e-05 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098475+0000", 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3002000000000003e-05 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16638 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1809257459999998, 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055689097, 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16638 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043234+0000", 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043237+0000", 2024-09-13T04:58:25.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6260000000000001e-06 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043240+0000", 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.627e-06 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098773+0000", 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055532519000000002 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098849+0000", 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6808000000000001e-05 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098921+0000", 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1835000000000006e-05 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16683 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1728748499999995, 2024-09-13T04:58:25.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055570913999999999, 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16683 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051285+0000", 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051294+0000", 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2239999999999995e-06 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051299+0000", 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0259999999999998e-06 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106697+0000", 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055398672000000003 2024-09-13T04:58:25.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106781+0000", 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3849000000000003e-05 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106854+0000", 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2683999999999997e-05 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16684 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1728543869999992, 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055767156999999998, 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16684 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051305+0000", 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.204e-06 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051313+0000", 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4830000000000002e-06 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051318+0000", 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6529999999999999e-06 2024-09-13T04:58:25.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106896+0000", 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055577731999999998 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106987+0000", 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0920000000000004e-05 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107071+0000", 2024-09-13T04:58:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3492999999999995e-05 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16685 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1728352379999993, 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055971087000000003, 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16685 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051322+0000", 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051325+0000", 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4499999999999998e-06 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051332+0000", 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2409999999999998e-06 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051337+0000", 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0499999999999999e-06 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107129+0000", 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055791855000000001 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107212+0000", 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3046000000000004e-05 2024-09-13T04:58:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107294+0000", 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1773000000000005e-05 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16689 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1715709739999998, 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694117000000001, 2024-09-13T04:58:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16689 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052589+0000", 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4420000000000001e-06 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052595+0000", 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8549999999999998e-06 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052600+0000", 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8169999999999998e-06 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108111+0000", 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055511660999999997 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108201+0000", 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9386000000000001e-05 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108281+0000", 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0338000000000002e-05 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16691 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1711547309999997, 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055651254999999997, 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16691 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053005+0000", 2024-09-13T04:58:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4490000000000002e-06 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053010+0000", 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2680000000000002e-06 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053014+0000", 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9729999999999999e-06 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108498+0000", 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055483769000000002 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108580+0000", 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1638999999999995e-05 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108654+0000", 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4548000000000005e-05 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.313 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.313 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.313 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:25.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:25.315 INFO:teuthology.orchestra.run.smithi001.stdout:1123 write 0xed27c21 thru 0xed375c9 (0xf9a9 bytes) 2024-09-13T04:58:25.315 INFO:teuthology.orchestra.run.smithi001.stdout:1124 write 0xd8e6f57 thru 0xd8ec6d4 (0x577e bytes) 2024-09-13T04:58:25.316 INFO:teuthology.orchestra.run.smithi001.stdout:1126 write 0xab65333 thru 0xab6d5d5 (0x82a3 bytes) 2024-09-13T04:58:25.316 INFO:teuthology.orchestra.run.smithi001.stdout:1127 write 0x148750c thru 0x148c090 (0x4b85 bytes) 2024-09-13T04:58:25.316 INFO:teuthology.orchestra.run.smithi001.stdout:1128 read 0x3fd619a thru 0x3fd78d4 (0x173b bytes) 2024-09-13T04:58:25.316 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.316 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.316 INFO:teuthology.orchestra.run.smithi001.stdout:1131 write 0x8d4c25d thru 0x8d4ff5a (0x3cfe bytes) 2024-09-13T04:58:25.316 INFO:teuthology.orchestra.run.smithi001.stdout:1132 writesame 0xb140a75 thru 0xb14e98f data_size 0x1(0xdf1b bytes) 2024-09-13T04:58:25.317 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:25.312+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:25.325 INFO:teuthology.orchestra.run.smithi001.stdout:1133 read 0x1ce6640 thru 0x1ceaa62 (0x4423 bytes) 2024-09-13T04:58:25.327 INFO:teuthology.orchestra.run.smithi001.stdout:1134 writesame 0x40f0a00 thru 0x40f13a3 data_size 0x2(0x9a4 bytes) 2024-09-13T04:58:25.334 INFO:teuthology.orchestra.run.smithi001.stdout:1136 read 0xaa73505 thru 0xaa75cb4 (0x27b0 bytes) 2024-09-13T04:58:25.335 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.336 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.336 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.336 INFO:teuthology.orchestra.run.smithi001.stdout:1141 writesame 0x5ef03b0 thru 0x5ef8fb8 data_size 0x1(0x8c09 bytes) 2024-09-13T04:58:25.357 INFO:teuthology.orchestra.run.smithi001.stdout:1142 read 0x17ddb2f thru 0x17e72e2 (0x97b4 bytes) 2024-09-13T04:58:25.358 INFO:teuthology.orchestra.run.smithi001.stdout:1143 write 0x5bf6bb6 thru 0x5bf6f7e (0x3c9 bytes) 2024-09-13T04:58:25.364 INFO:teuthology.orchestra.run.smithi001.stdout:1144 write 0x2aed92f thru 0x2afbbbc (0xe28e bytes) 2024-09-13T04:58:25.376 INFO:teuthology.orchestra.run.smithi001.stdout:1145 read 0xce94122 thru 0xce9d26d (0x914c bytes) 2024-09-13T04:58:25.377 INFO:teuthology.orchestra.run.smithi001.stdout:1146 write 0x112f9a4 thru 0x113f0b4 (0xf711 bytes) 2024-09-13T04:58:25.382 INFO:teuthology.orchestra.run.smithi001.stdout:1147 writesame 0xe3c95c8 thru 0xe3cc83f data_size 0x55(0x3278 bytes) 2024-09-13T04:58:25.389 INFO:teuthology.orchestra.run.smithi001.stdout:1148 punch from 0x465563a to 0x46558ad, (0x273 bytes) 2024-09-13T04:58:25.389 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xee25360 2024-09-13T04:58:25.389 INFO:teuthology.orchestra.run.smithi001.stdout:1149 trunc from 0xed375ca to 0xee25360 2024-09-13T04:58:25.400 INFO:teuthology.orchestra.run.smithi001.stdout:1154 writesame 0xc8f6e96 thru 0xc9031af data_size 0x22(0xc31a bytes) 2024-09-13T04:58:25.408 INFO:teuthology.orchestra.run.smithi001.stdout:1155 write 0xbbc016c thru 0xbbc33d3 (0x3268 bytes) 2024-09-13T04:58:25.413 INFO:teuthology.orchestra.run.smithi001.stdout:1156 write 0x6cd049b thru 0x6cda902 (0xa468 bytes) 2024-09-13T04:58:25.415 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:25.412+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:25.423 INFO:teuthology.orchestra.run.smithi001.stdout:1158 write 0x811bafa thru 0x811d81c (0x1d23 bytes) 2024-09-13T04:58:25.429 INFO:teuthology.orchestra.run.smithi001.stdout:1159 write 0xe916763 thru 0xe924296 (0xdb34 bytes) 2024-09-13T04:58:25.437 INFO:teuthology.orchestra.run.smithi001.stdout:1160 write 0xc52cbe7 thru 0xc53a34f (0xd769 bytes) 2024-09-13T04:58:25.442 INFO:teuthology.orchestra.run.smithi001.stdout:1161 writesame 0x861aac7 thru 0x8620e03 data_size 0x1(0x633d bytes) 2024-09-13T04:58:25.446 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:25.446 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:25.447 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:25.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:25.481 INFO:teuthology.orchestra.run.smithi001.stdout:1162 trunc from 0xee25360 to 0x9d61ce9 2024-09-13T04:58:25.517 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:25.516+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:25.521 INFO:teuthology.orchestra.run.smithi001.stdout:1164 write 0x9733621 thru 0x973d03a (0x9a1a bytes) 2024-09-13T04:58:25.531 INFO:teuthology.orchestra.run.smithi001.stdout:1165 read 0x11c1d6b thru 0x11d0bac (0xee42 bytes) 2024-09-13T04:58:25.532 INFO:teuthology.orchestra.run.smithi001.stdout:1166 read 0x6ec538 thru 0x6f8b51 (0xc61a bytes) 2024-09-13T04:58:25.532 INFO:teuthology.orchestra.run.smithi001.stdout:1167 writesame 0x23848d0 thru 0x2394493 data_size 0xc(0xfbc4 bytes) 2024-09-13T04:58:25.543 INFO:teuthology.orchestra.run.smithi001.stdout:1168 writesame 0x2fcea54 thru 0x2fd9554 data_size 0x1(0xab01 bytes) 2024-09-13T04:58:25.600 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.604 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:25.605 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:25.617 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:25.616+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:25.619 INFO:teuthology.orchestra.run.smithi001.stdout:1170 trunc from 0x9d61ce9 to 0x6576b7 2024-09-13T04:58:25.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:25.713 INFO:teuthology.orchestra.run.smithi001.stdout:1171 write 0x3d0d164 thru 0x3d1bf9d (0xee3a bytes) 2024-09-13T04:58:25.718 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:25.716+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:25.747 INFO:teuthology.orchestra.run.smithi001.stdout:1172 write 0xc649e06 thru 0xc64a2ab (0x4a6 bytes) 2024-09-13T04:58:25.754 INFO:teuthology.orchestra.run.smithi001.stdout:1173 write 0x6815548 thru 0x6818860 (0x3319 bytes) 2024-09-13T04:58:25.760 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.760 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.760 INFO:teuthology.orchestra.run.smithi001.stdout:1177 writesame 0x41937e3 thru 0x41a34ce data_size 0x2(0xfcec bytes) 2024-09-13T04:58:25.813 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:13769 3.5 3:a55a4cd2:::rbd_data.112a2c87bb71.0000000000000120:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.065790906, 2024-09-13T04:58:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007083972, 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 13769 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743806+0000", 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743805+0000", 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743810+0000", 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0540000000000002e-06 2024-09-13T04:58:25.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743823+0000", 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.275e-05 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743831+0000", 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6750000000000002e-06 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743877+0000", 2024-09-13T04:58:25.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6597000000000001e-05 2024-09-13T04:58:25.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.743956+0000", 2024-09-13T04:58:25.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8732000000000004e-05 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.744121+0000", 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016536900000000001 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.745356+0000", 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001234824 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750813+0000", 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00545671 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750854+0000", 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1309e-05 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:15.750890+0000", 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5927999999999999e-05 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.3484381019999994, 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:25.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:58:25.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:58:25.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:58:25.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:58:25.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28802 3.16 3:6d9fd88d:::rbd_object_map.112a547f4795:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.087971+0000", 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.7216264360000002, 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060163980000000001, 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28802 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.087971+0000", 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.087971+0000", 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.087968+0000", 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.087985+0000", 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.696e-05 2024-09-13T04:58:25.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088018+0000", 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3485000000000002e-05 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088042+0000", 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3618999999999999e-05 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088185+0000", 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014307 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088450+0000", 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026445400000000001 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.089540+0000", 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010899270000000001 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.093626+0000", 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040866440000000004 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.093739+0000", 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112417 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.093883+0000", 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000144714 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.093987+0000", 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000103805 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:28803 3.16 3:6d9fd88d:::rbd_object_map.112a547f4795:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.088577+0000", 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.721019794, 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066222870000000001, 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 28803 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088577+0000", 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088577+0000", 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088575+0000", 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088589+0000", 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4249e-05 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088617+0000", 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7121999999999999e-05 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.088630+0000", 2024-09-13T04:58:25.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.348e-05 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.090888+0000", 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022581559999999999 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.091135+0000", 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024649900000000001 2024-09-13T04:58:25.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.092384+0000", 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012491729999999999 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.094550+0000", 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021664639999999999 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.094997+0000", 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044710300000000002 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.095137+0000", 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013970000000000001 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.095200+0000", 2024-09-13T04:58:25.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2557000000000004e-05 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30534 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823146+0000", 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9864515010000003, 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083493909999999994, 2024-09-13T04:58:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30534 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823146+0000", 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823146+0000", 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823144+0000", 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823153+0000", 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7660000000000004e-06 2024-09-13T04:58:25.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823172+0000", 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8374999999999999e-05 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823182+0000", 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0732e-05 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823237+0000", 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4301000000000001e-05 2024-09-13T04:58:25.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823324+0000", 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7865e-05 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823781+0000", 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045668299999999999 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827196+0000", 2024-09-13T04:58:25.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034147930000000002 2024-09-13T04:58:25.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831407+0000", 2024-09-13T04:58:25.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042107070000000002 2024-09-13T04:58:25.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831456+0000", 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9398999999999999e-05 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831495+0000", 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9016e-05 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30535 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823219+0000", 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9863782959999998, 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010033459999999999, 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.847 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30535 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823219+0000", 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823219+0000", 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823218+0000", 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823225+0000", 2024-09-13T04:58:25.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0809999999999996e-06 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823238+0000", 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2788e-05 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823247+0000", 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824276+0000", 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010286920000000001 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824357+0000", 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1415999999999994e-05 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824623+0000", 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026604299999999999 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827205+0000", 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002581421 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833190+0000", 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059849150000000004 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833219+0000", 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9339000000000001e-05 2024-09-13T04:58:25.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833252+0000", 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3249e-05 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30536 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823272+0000", 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9863256570000001, 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010117332999999999, 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30536 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823272+0000", 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823272+0000", 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823271+0000", 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823276+0000", 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823288+0000", 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2014999999999999e-05 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823297+0000", 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4700000000000002e-06 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824906+0000", 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001609463 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.824974+0000", 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8053999999999996e-05 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.825237+0000", 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026292399999999999 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828888+0000", 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036508140000000001 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833301+0000", 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004413152 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833348+0000", 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7144999999999997e-05 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833389+0000", 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0586000000000001e-05 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30539 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823354+0000", 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9862435959999996, 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012316781000000001, 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30539 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.861 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823354+0000", 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823354+0000", 2024-09-13T04:58:25.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823352+0000", 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823359+0000", 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.63e-06 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823372+0000", 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2948e-05 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823381+0000", 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5140000000000001e-06 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.825528+0000", 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002147693 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.825575+0000", 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6873999999999999e-05 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.825839+0000", 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026439299999999997 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828899+0000", 2024-09-13T04:58:25.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030599809999999998 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835606+0000", 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067060399999999999 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835634+0000", 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8365999999999999e-05 2024-09-13T04:58:25.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835670+0000", 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6464999999999997e-05 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30540 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823479+0000", 2024-09-13T04:58:25.868 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9861176929999997, 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012343474, 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30540 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823479+0000", 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823479+0000", 2024-09-13T04:58:25.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823478+0000", 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823487+0000", 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4030000000000007e-06 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823517+0000", 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9878000000000001e-05 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823527+0000", 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0536999999999999e-05 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.826173+0000", 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026460680000000001 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.826302+0000", 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012925 2024-09-13T04:58:25.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.826743+0000", 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044028999999999997 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830129+0000", 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003385805 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835719+0000", 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055906890000000002 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835768+0000", 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9162999999999997e-05 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835823+0000", 2024-09-13T04:58:25.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4552999999999998e-05 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30541 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823556+0000", 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9860409570000002, 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012393184, 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30541 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823556+0000", 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823556+0000", 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823555+0000", 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823562+0000", 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9480000000000002e-06 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823574+0000", 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2047e-05 2024-09-13T04:58:25.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823582+0000", 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9249999999999995e-06 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827251+0000", 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036688440000000001 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827342+0000", 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0674000000000006e-05 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827634+0000", 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000292358 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830131+0000", 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024971989999999999 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835874+0000", 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005742601 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835912+0000", 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8052000000000001e-05 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.835949+0000", 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7370000000000003e-05 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.880 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30542 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823605+0000", 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9859917720000002, 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012485001000000001, 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30542 2024-09-13T04:58:25.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823605+0000", 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823605+0000", 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823605+0000", 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823611+0000", 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4060000000000002e-06 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823622+0000", 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1221e-05 2024-09-13T04:58:25.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823630+0000", 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8110000000000003e-06 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.827943+0000", 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043126789999999998 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828041+0000", 2024-09-13T04:58:25.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8475999999999997e-05 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828478+0000", 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00043719599999999999 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831354+0000", 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028756250000000001 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836007+0000", 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046533980000000004 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836055+0000", 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8007000000000003e-05 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836090+0000", 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4929999999999999e-05 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30543 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823638+0000", 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9859596489999998, 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01329027, 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30543 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823638+0000", 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823638+0000", 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823637+0000", 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823642+0000", 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7709999999999997e-06 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823655+0000", 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3414000000000001e-05 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823663+0000", 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5179999999999998e-06 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.828946+0000", 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052829060000000004 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829034+0000", 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7979000000000006e-05 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829300+0000", 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026641600000000001 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831357+0000", 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020569590000000001 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836829+0000", 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054724259999999999 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836893+0000", 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3313999999999997e-05 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836928+0000", 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5157000000000002e-05 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.892 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30544 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823686+0000", 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9859107900000001, 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013377933, 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30544 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823686+0000", 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823686+0000", 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823685+0000", 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823691+0000", 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0009999999999998e-06 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823702+0000", 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0672999999999999e-05 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823710+0000", 2024-09-13T04:58:25.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5939999999999996e-06 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829632+0000", 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059225720000000001 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829683+0000", 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1242000000000002e-05 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.829886+0000", 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020243799999999999 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.832049+0000", 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021627619999999999 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.836980+0000", 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004930963 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.837028+0000", 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8659999999999998e-05 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.837064+0000", 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6127999999999997e-05 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30545 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823717+0000", 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9858802559999997, 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01503137, 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30545 2024-09-13T04:58:25.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823717+0000", 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823717+0000", 2024-09-13T04:58:25.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823716+0000", 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823721+0000", 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1109999999999999e-06 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823730+0000", 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.037e-06 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823754+0000", 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3283000000000001e-05 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830192+0000", 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064381430000000003 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830290+0000", 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8552000000000001e-05 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830560+0000", 2024-09-13T04:58:25.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026991899999999997 2024-09-13T04:58:25.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.832051+0000", 2024-09-13T04:58:25.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001490901 2024-09-13T04:58:25.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838665+0000", 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006613687 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838711+0000", 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6536000000000003e-05 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838748+0000", 2024-09-13T04:58:25.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6788000000000001e-05 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30546 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823776+0000", 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9858213429999996, 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015108465, 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30546 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823776+0000", 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823776+0000", 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823775+0000", 2024-09-13T04:58:25.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823780+0000", 2024-09-13T04:58:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1039999999999998e-06 2024-09-13T04:58:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823807+0000", 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6262e-05 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823817+0000", 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8020000000000003e-06 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830869+0000", 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070525789999999998 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.830922+0000", 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3183000000000002e-05 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831111+0000", 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018851100000000001 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.832056+0000", 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094489200000000002 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838811+0000", 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067551410000000001 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838852+0000", 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0874000000000002e-05 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.838884+0000", 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2598999999999997e-05 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30547 3.5 3:a20d0956:::rbd_object_map.112a1dc60142:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.823825+0000", 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9857725469999998, 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01570467, 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30547 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823825+0000", 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823825+0000", 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823824+0000", 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823830+0000", 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.443e-06 2024-09-13T04:58:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823840+0000", 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0132000000000001e-05 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.823846+0000", 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6919999999999999e-06 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831534+0000", 2024-09-13T04:58:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0076881500000000004 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831593+0000", 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9419000000000001e-05 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.831828+0000", 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000234229 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.832476+0000", 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00064819699999999999 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839440+0000", 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069641629999999998 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839490+0000", 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9685000000000003e-05 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839529+0000", 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9578e-05 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30552 3.5 3:a4e32c5e:::rbd_data.112a1dc60142.00000000000000c5:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:20.833493+0000", 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.976103878, 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064428080000000004, 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30552 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833493+0000", 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833493+0000", 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833492+0000", 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833498+0000", 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8719999999999999e-06 2024-09-13T04:58:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833509+0000", 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0664e-05 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833514+0000", 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2349999999999999e-06 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833547+0000", 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3285999999999999e-05 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833640+0000", 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2330000000000005e-05 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.833750+0000", 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011045899999999999 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.834144+0000", 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039349600000000002 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839822+0000", 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005678336 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839900+0000", 2024-09-13T04:58:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8448000000000005e-05 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:20.839936+0000", 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5697000000000001e-05 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:30978 3.5 3:a140ae6c:::rbd_data.112a1dc60142.0000000000000084:head [set-alloc-hint object_size 1048576 write_size 1048576,write 690788~37801] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:21.006253+0000", 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.8033438640000004, 2024-09-13T04:58:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062508579999999998, 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 30978 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006253+0000", 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006253+0000", 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006252+0000", 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006271+0000", 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8952000000000001e-05 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006286+0000", 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4960000000000001e-05 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006294+0000", 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9999999999999996e-06 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006327+0000", 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3285999999999999e-05 2024-09-13T04:58:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006405+0000", 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7645000000000006e-05 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 6", 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.006484+0000", 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8464e-05 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.008310+0000", 2024-09-13T04:58:25.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018266860000000001 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.012456+0000", 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041456770000000004 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.012484+0000", 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8204e-05 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:21.012504+0000", 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0075e-05 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:37637 3.11 e41/24 3:88b77d11:::rbd_data.112a66af081a.0000000000000000:head v 41'72, mlcod=41'70)", 2024-09-13T04:58:25.931 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.4542391459999999, 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027255564999999999, 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:25.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355356+0000", 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356532+0000", 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001176599 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356563+0000", 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0916e-05 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356576+0000", 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3293e-05 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356700+0000", 2024-09-13T04:58:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123779 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356746+0000", 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6242000000000002e-05 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382519+0000", 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025772856 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382614+0000", 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4315999999999996e-05 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:37638 3.16 3:6fe11eb9:::rbd_data.112a66af081a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1420443] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.356680+0000", 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.4529168129999999, 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016175605999999999, 2024-09-13T04:58:25.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 37638 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356680+0000", 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356680+0000", 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356678+0000", 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.357757+0000", 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001079244 2024-09-13T04:58:25.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.357775+0000", 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7867e-05 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.357784+0000", 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0769999999999997e-06 2024-09-13T04:58:25.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.357843+0000", 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8866999999999997e-05 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.358006+0000", 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016324500000000001 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.358282+0000", 2024-09-13T04:58:25.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000275973 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.372711+0000", 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014429348 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.372744+0000", 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2848000000000001e-05 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.372810+0000", 2024-09-13T04:58:25.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6314000000000002e-05 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.372856+0000", 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5568000000000001e-05 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:25.942 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:25.943 INFO:teuthology.orchestra.run.smithi001.stdout:1178 writesame 0xd27cf80 thru 0xd28275f data_size 0x4c(0x57e0 bytes) 2024-09-13T04:58:25.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:25.945 INFO:teuthology.orchestra.run.smithi001.stdout:1180 trunc from 0xd282760 to 0xc30e6b0 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1182 write 0x4caa4ed thru 0x4cae8b2 (0x43c6 bytes) 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1184 flatten 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1185 punch from 0x93a019d to 0x93af916, (0xf779 bytes) 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1186 trunc from 0xc30e6b0 to 0xbe41040 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1187 read 0x54a2e4 thru 0x54aac9 (0x7e6 bytes) 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1188 write 0x7b6970b thru 0x7b76db5 (0xd6ab bytes) 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1190 trunc from 0xbe41040 to 0xd828577 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1191 writesame 0xe19839d thru 0xe1a207e data_size 0x2b(0x9ce2 bytes) 2024-09-13T04:58:25.946 INFO:teuthology.orchestra.run.smithi001.stdout:1192 writesame 0x247fcb5 thru 0x248d789 data_size 0x97(0xdad5 bytes) 2024-09-13T04:58:25.947 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:25.944+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:25.970 INFO:teuthology.orchestra.run.smithi001.stdout:1193 trunc from 0xe1a207f to 0x396a3b6 2024-09-13T04:58:26.014 INFO:teuthology.orchestra.run.smithi001.stdout:1194 writesame 0x77ab28c thru 0x77b07f1 data_size 0x11(0x5566 bytes) 2024-09-13T04:58:26.015 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.024 INFO:teuthology.orchestra.run.smithi001.stdout:1196 trunc from 0x77b07f2 to 0x32c8ad3 2024-09-13T04:58:26.038 INFO:teuthology.orchestra.run.smithi001.stdout:1198 read 0x191986e thru 0x191c652 (0x2de5 bytes) 2024-09-13T04:58:26.038 INFO:teuthology.orchestra.run.smithi001.stdout:1199 read 0x17dd001 thru 0x17e2755 (0x5755 bytes) 2024-09-13T04:58:26.045 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:26.044+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:26.072 INFO:teuthology.orchestra.run.smithi001.stdout:1200 writesame 0xbf10c29 thru 0xbf18573 data_size 0x1(0x794b bytes) 2024-09-13T04:58:26.072 INFO:teuthology.orchestra.run.smithi001.stdout:1201 read 0x4b9c2fd thru 0x4bab767 (0xf46b bytes) 2024-09-13T04:58:26.079 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:26.079 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:26.079 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:26.079 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:26.086 INFO:teuthology.orchestra.run.smithi001.stdout:1203 write 0xc236d59 thru 0xc23f9e6 (0x8c8e bytes) 2024-09-13T04:58:26.093 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:26.104 INFO:teuthology.orchestra.run.smithi001.stdout:1205 trunc from 0xc23f9e7 to 0x777c1db 2024-09-13T04:58:26.126 INFO:teuthology.orchestra.run.smithi001.stdout:1206 punch from 0x3d83a82 to 0x3d8cfe8, (0x9566 bytes) 2024-09-13T04:58:26.126 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.126 INFO:teuthology.orchestra.run.smithi001.stdout:1209 punch from 0x2bc69b8 to 0x2bd0933, (0x9f7b bytes) 2024-09-13T04:58:26.126 INFO:teuthology.orchestra.run.smithi001.stdout:1212 write 0x8ebf53 thru 0x8f0f26 (0x4fd4 bytes) 2024-09-13T04:58:26.132 INFO:teuthology.orchestra.run.smithi001.stdout:1213 read 0x94c32f thru 0x959abe (0xd790 bytes) 2024-09-13T04:58:26.132 INFO:teuthology.orchestra.run.smithi001.stdout:1214 write 0x5254ec8 thru 0x52581fa (0x3333 bytes) 2024-09-13T04:58:26.140 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.145 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:26.144+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:26.146 INFO:teuthology.orchestra.run.smithi001.stdout:1216 trunc from 0x777c1db to 0xa279463 2024-09-13T04:58:26.158 INFO:teuthology.orchestra.run.smithi001.stdout:1217 read 0xc4fc52 thru 0xc4fe98 (0x247 bytes) 2024-09-13T04:58:26.158 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.159 INFO:teuthology.orchestra.run.smithi001.stdout:1221 write 0x7e6c494 thru 0x7e7bd62 (0xf8cf bytes) 2024-09-13T04:58:26.172 INFO:teuthology.orchestra.run.smithi001.stdout:1222 writesame 0x5ea89e2 thru 0x5eb4334 data_size 0x13(0xb953 bytes) 2024-09-13T04:58:26.181 INFO:teuthology.orchestra.run.smithi001.stdout:1223 read 0x7d32d9 thru 0x7dc474 (0x919c bytes) 2024-09-13T04:58:26.183 INFO:teuthology.orchestra.run.smithi001.stdout:1224 read 0x307de41 thru 0x30876d9 (0x9899 bytes) 2024-09-13T04:58:26.183 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.183 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.183 INFO:teuthology.orchestra.run.smithi001.stdout:1229 write 0x93151e8 thru 0x9324a42 (0xf85b bytes) 2024-09-13T04:58:26.198 INFO:teuthology.orchestra.run.smithi001.stdout:1230 write 0x41f1a99 thru 0x41f1df5 (0x35d bytes) 2024-09-13T04:58:26.201 INFO:teuthology.orchestra.run.smithi001.stdout:1231 read 0x33b8dbe thru 0x33c483b (0xba7e bytes) 2024-09-13T04:58:26.218 INFO:teuthology.orchestra.run.smithi001.stdout:1232 writesame 0xbb1f3b0 thru 0xbb2b3b2 data_size 0x57(0xc003 bytes) 2024-09-13T04:58:26.218 INFO:teuthology.orchestra.run.smithi001.stdout:1233 writesame 0x910ffa6 thru 0x911d44b data_size 0x81(0xd4a6 bytes) 2024-09-13T04:58:26.226 INFO:teuthology.orchestra.run.smithi001.stdout:1234 read 0x55a8888 thru 0x55aa790 (0x1f09 bytes) 2024-09-13T04:58:26.227 INFO:teuthology.orchestra.run.smithi001.stdout:1236 write 0x8e39e94 thru 0x8e48460 (0xe5cd bytes) 2024-09-13T04:58:26.233 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:26.233 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:26.233 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:26.233 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:26.233 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:26.241 INFO:teuthology.orchestra.run.smithi001.stdout:1237 write 0x688abca thru 0x6896fae (0xc3e5 bytes) 2024-09-13T04:58:26.246 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:26.244+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:26.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:26.253 INFO:teuthology.orchestra.run.smithi001.stdout:1239 punch from 0xa1ca071 to 0xa1cc213, (0x21a2 bytes) 2024-09-13T04:58:26.253 INFO:teuthology.orchestra.run.smithi001.stdout:1241 punch from 0x57737a8 to 0x577dd56, (0xa5ae bytes) 2024-09-13T04:58:26.253 INFO:teuthology.orchestra.run.smithi001.stdout:1242 read 0x2416dd6 thru 0x242478b (0xd9b6 bytes) 2024-09-13T04:58:26.254 INFO:teuthology.orchestra.run.smithi001.stdout:1243 punch from 0x470b241 to 0x47136ce, (0x848d bytes) 2024-09-13T04:58:26.261 INFO:teuthology.orchestra.run.smithi001.stdout:1244 trunc from 0xbb2b3b3 to 0x83d39b7 2024-09-13T04:58:26.292 INFO:teuthology.orchestra.run.smithi001.stdout:1246 write 0x718a05c thru 0x718b543 (0x14e8 bytes) 2024-09-13T04:58:26.313 INFO:teuthology.orchestra.run.smithi001.stdout:1247 write 0xb5ac5f0 thru 0xb5adce0 (0x16f1 bytes) 2024-09-13T04:58:26.319 INFO:teuthology.orchestra.run.smithi001.stdout:1248 read 0xa47d384 thru 0xa4820e6 (0x4d63 bytes) 2024-09-13T04:58:26.319 INFO:teuthology.orchestra.run.smithi001.stdout:1249 writesame 0x61646d0 thru 0x61670a7 data_size 0x34(0x29d8 bytes) 2024-09-13T04:58:26.326 INFO:teuthology.orchestra.run.smithi001.stdout:1251 writesame 0x5a66cdf thru 0x5a74da4 data_size 0x1(0xe0c6 bytes) 2024-09-13T04:58:26.347 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:26.344+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:26.377 INFO:teuthology.orchestra.run.smithi001.stdout:1252 punch from 0x59af7e9 to 0x59ba914, (0xb12b bytes) 2024-09-13T04:58:26.378 INFO:teuthology.orchestra.run.smithi001.stdout:1253 write 0x1379d82 thru 0x1387cd8 (0xdf57 bytes) 2024-09-13T04:58:26.384 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.218368523999999, 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:58:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:15554 3.a 3:54a75321:::rbd_data.112af6c12117.0000000000000002:6 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.357000358000001, 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047745517000000001, 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 15554 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025346+0000", 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025348+0000", 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1900000000000002e-06 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025355+0000", 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7850000000000003e-06 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025360+0000", 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6789999999999996e-06 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.067347+0000", 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041987402 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.072996+0000", 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056486540000000003 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.073092+0000", 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6151999999999997e-05 2024-09-13T04:58:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.215927864999999, 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:58:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:58:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17859 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1458176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.091559986, 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047596271000000002, 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17859 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290789+0000", 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290794+0000", 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7610000000000004e-06 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290799+0000", 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4889999999999997e-06 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333438+0000", 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042639200000000002 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333513+0000", 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5172000000000005e-05 2024-09-13T04:58:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.338383+0000", 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048697929999999999 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17891 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294255+0000", 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.08809233, 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045855475999999999, 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17891 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294255+0000", 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294255+0000", 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294254+0000", 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294256+0000", 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1849999999999998e-06 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294262+0000", 2024-09-13T04:58:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.801e-06 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294267+0000", 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340012+0000", 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045745134999999999 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340060+0000", 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8435e-05 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340110+0000", 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9546000000000003e-05 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17892 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1748992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294271+0000", 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.088075829999999, 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045962984999999998, 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17892 2024-09-13T04:58:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294271+0000", 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294271+0000", 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294271+0000", 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294273+0000", 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1349999999999999e-06 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294279+0000", 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8850000000000004e-06 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294283+0000", 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.622e-06 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340137+0000", 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045853440000000002 2024-09-13T04:58:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340181+0000", 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4665e-05 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340234+0000", 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2593000000000003e-05 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17893 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.088050573, 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046056773000000002, 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17893 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294298+0000", 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5000000000000002e-06 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294307+0000", 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294312+0000", 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.451e-06 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340260+0000", 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045947904999999997 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340302+0000", 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1650999999999998e-05 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340353+0000", 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1208000000000002e-05 2024-09-13T04:58:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17894 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.088030067, 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046155512000000003, 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17894 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294316+0000", 2024-09-13T04:58:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294319+0000", 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2390000000000001e-06 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294325+0000", 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0340000000000002e-06 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294329+0000", 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7509999999999999e-06 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340378+0000", 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046048248 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340421+0000", 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3627000000000001e-05 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340472+0000", 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17895 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.088013283, 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046268562999999999, 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17895 2024-09-13T04:58:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294333+0000", 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294335+0000", 2024-09-13T04:58:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294341+0000", 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9939999999999997e-06 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294346+0000", 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7500000000000003e-06 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340508+0000", 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046161354000000002 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340550+0000", 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2962e-05 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340602+0000", 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1665000000000001e-05 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17896 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1912832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.087996535, 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046367673999999998, 2024-09-13T04:58:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17896 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294352+0000", 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.17e-06 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294358+0000", 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2249999999999997e-06 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294363+0000", 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6990000000000002e-06 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340627+0000", 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046263836000000003 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340668+0000", 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0638e-05 2024-09-13T04:58:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340718+0000", 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0458999999999999e-05 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17897 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1953792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.087979726, 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046474117000000002, 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17897 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294369+0000", 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294375+0000", 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9880000000000001e-06 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294381+0000", 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7939999999999999e-06 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340744+0000", 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046362962000000001 2024-09-13T04:58:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340786+0000", 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2311000000000002e-05 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340841+0000", 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5192999999999998e-05 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17898 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1994752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.087961663, 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046575812000000001, 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17898 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294387+0000", 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1600000000000001e-06 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294395+0000", 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.52e-06 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294400+0000", 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3149999999999999e-06 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340868+0000", 2024-09-13T04:58:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046467893000000003 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340911+0000", 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3340000000000002e-05 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340961+0000", 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9934e-05 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17899 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2035712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.087943004, 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691141999999998, 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17899 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294406+0000", 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294411+0000", 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294416+0000", 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5600000000000004e-06 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340987+0000", 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046570645000000001 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341045+0000", 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8065000000000001e-05 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341095+0000", 2024-09-13T04:58:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0302000000000001e-05 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17900 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2076672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.087926541, 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046796380999999998, 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17900 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294422+0000", 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9609999999999999e-06 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294428+0000", 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9290000000000003e-06 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294432+0000", 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4190000000000004e-06 2024-09-13T04:58:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341123+0000", 2024-09-13T04:58:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046690597 2024-09-13T04:58:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341165+0000", 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2289000000000001e-05 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341217+0000", 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1484999999999999e-05 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17901 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.087910511, 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046918842000000002, 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17901 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294438+0000", 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9599999999999999e-06 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294444+0000", 2024-09-13T04:58:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7740000000000001e-06 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294448+0000", 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5210000000000004e-06 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341242+0000", 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046794009999999997 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341288+0000", 2024-09-13T04:58:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.596e-05 2024-09-13T04:58:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341355+0000", 2024-09-13T04:58:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6931e-05 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17904 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.08789447, 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006963999999998, 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17904 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294454+0000", 2024-09-13T04:58:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.979e-06 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294460+0000", 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7899999999999996e-06 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294464+0000", 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4139999999999996e-06 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341362+0000", 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046897498000000003 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341407+0000", 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4839000000000002e-05 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341459+0000", 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2719000000000001e-05 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17911 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.087878571999999, 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047111301000000001, 2024-09-13T04:58:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17911 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294470+0000", 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9769999999999999e-06 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294476+0000", 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.925e-06 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294480+0000", 2024-09-13T04:58:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5390000000000001e-06 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341487+0000", 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006829 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341530+0000", 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2503000000000003e-05 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341580+0000", 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9848000000000001e-05 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17914 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2048000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.08786252, 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047212684999999997, 2024-09-13T04:58:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17914 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294486+0000", 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0899999999999999e-06 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294492+0000", 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1920000000000002e-06 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294497+0000", 2024-09-13T04:58:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5399999999999997e-06 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341606+0000", 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047108783000000001 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341648+0000", 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1878999999999997e-05 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341697+0000", 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9521999999999998e-05 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17916 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2088960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.087845713, 2024-09-13T04:58:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047303235999999999, 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17916 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294506+0000", 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1880000000000001e-06 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294518+0000", 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1802e-05 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294523+0000", 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7219999999999999e-06 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341713+0000", 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190086999999999 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341755+0000", 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2530000000000001e-05 2024-09-13T04:58:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341804+0000", 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9245e-05 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9266955709999998, 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:58:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:58:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:58:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:26.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:26.493 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:58:26.494 INFO:teuthology.orchestra.run.smithi001.stdout:1254 read 0x604dbce thru 0x60565f4 (0x8a27 bytes) 2024-09-13T04:58:26.494 INFO:teuthology.orchestra.run.smithi001.stdout:1255 write 0x6778f0e thru 0x677abd6 (0x1cc9 bytes) 2024-09-13T04:58:26.495 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:1256 punch from 0x1a62d4d to 0x1a6ac77, (0x7f2a bytes) 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:1258 read 0x8c50eec thru 0x8c5f680 (0xe795 bytes) 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:1259 write 0x6bacd39 thru 0x6bb7632 (0xa8fa bytes) 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:1260 read 0x30f49a4 thru 0x30f8549 (0x3ba6 bytes) 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:1261 read 0x73a1b95 thru 0x73ad287 (0xb6f3 bytes) 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:1262 read 0x4c3f2e0 thru 0x4c4ee85 (0xfba6 bytes) 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:1265 writesame 0xc97ab73 thru 0xc97e38e data_size 0x2a(0x381c bytes) 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.497 INFO:teuthology.orchestra.run.smithi001.stdout:1267 write 0xe89b6ca thru 0xe8a6ac4 (0xb3fb bytes) 2024-09-13T04:58:26.498 INFO:teuthology.orchestra.run.smithi001.stdout:1270 punch from 0xa056ae4 to 0xa05776c, (0xc88 bytes) 2024-09-13T04:58:26.498 INFO:teuthology.orchestra.run.smithi001.stdout:1271 trunc from 0xe8a6ac5 to 0x8b9ce3a 2024-09-13T04:58:26.498 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:26.492+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:26.505 INFO:teuthology.orchestra.run.smithi001.stdout:1272 punch from 0x4e2ef7c to 0x4e38bd8, (0x9c5c bytes) 2024-09-13T04:58:26.516 INFO:teuthology.orchestra.run.smithi001.stdout:1273 trunc from 0x8b9ce3a to 0x439b093 2024-09-13T04:58:26.572 INFO:teuthology.orchestra.run.smithi001.stdout:1276 write 0xcec98d8 thru 0xcecfe3b (0x6564 bytes) 2024-09-13T04:58:26.578 INFO:teuthology.orchestra.run.smithi001.stdout:1277 trunc from 0xcecfe3c to 0xc9b1bd9 2024-09-13T04:58:26.595 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:26.592+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:26.615 INFO:teuthology.orchestra.run.smithi001.stdout:1278 write 0xd8fe428 thru 0xd90caf0 (0xe6c9 bytes) 2024-09-13T04:58:26.623 INFO:teuthology.orchestra.run.smithi001.stdout:1279 punch from 0x564b69c to 0x5654257, (0x8bbb bytes) 2024-09-13T04:58:26.623 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.623 INFO:teuthology.orchestra.run.smithi001.stdout:1281 writesame 0xd8d12d8 thru 0xd8da381 data_size 0x1(0x90aa bytes) 2024-09-13T04:58:26.634 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:26.634 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:26.634 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:26.648 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:26.655 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.664 INFO:teuthology.orchestra.run.smithi001.stdout:1285 writesame 0xdcb6804 thru 0xdcc260a data_size 0x1(0xbe07 bytes) 2024-09-13T04:58:26.664 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.664 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.664 INFO:teuthology.orchestra.run.smithi001.stdout:1289 write 0x24aba30 thru 0x24b5fc7 (0xa598 bytes) 2024-09-13T04:58:26.668 INFO:teuthology.orchestra.run.smithi001.stdout:1290 punch from 0x29b6d28 to 0x29bcd7a, (0x6052 bytes) 2024-09-13T04:58:26.668 INFO:teuthology.orchestra.run.smithi001.stdout:1291 write 0xd72fae5 thru 0xd73f935 (0xfe51 bytes) 2024-09-13T04:58:26.674 INFO:teuthology.orchestra.run.smithi001.stdout:1292 punch from 0x6a69df3 to 0x6a79d67, (0xff74 bytes) 2024-09-13T04:58:26.674 INFO:teuthology.orchestra.run.smithi001.stdout:1293 write 0x8ab659a thru 0x8ab9552 (0x2fb9 bytes) 2024-09-13T04:58:26.684 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:26.684 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:26.688 INFO:teuthology.orchestra.run.smithi001.stdout:1294 trunc from 0xdcc260b to 0x625d081 2024-09-13T04:58:26.696 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:26.692+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:26.701 INFO:teuthology.orchestra.run.smithi001.stdout:1295 read 0x7156d thru 0x7eca2 (0xd736 bytes) 2024-09-13T04:58:26.701 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:26.702 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:26.702 INFO:teuthology.orchestra.run.smithi001.stdout:1296 read 0x425c047 thru 0x425e920 (0x28da bytes) 2024-09-13T04:58:26.720 INFO:teuthology.orchestra.run.smithi001.stdout:1297 write 0xdc529d0 thru 0xdc537e4 (0xe15 bytes) 2024-09-13T04:58:26.724 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.724 INFO:teuthology.orchestra.run.smithi001.stdout:1299 write 0x3f0f006 thru 0x3f0fc6e (0xc69 bytes) 2024-09-13T04:58:26.724 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:26.725 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:26.728 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.736 INFO:teuthology.orchestra.run.smithi001.stdout:1301 writesame 0xe22764b thru 0xe231b8c data_size 0x21(0xa542 bytes) 2024-09-13T04:58:26.736 INFO:teuthology.orchestra.run.smithi001.stdout:1302 write 0xbd6a5b7 thru 0xbd7114e (0x6b98 bytes) 2024-09-13T04:58:26.741 INFO:teuthology.orchestra.run.smithi001.stdout:1303 writesame 0x2899580 thru 0x28a7760 data_size 0x4b(0xe1e1 bytes) 2024-09-13T04:58:26.742 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:26.742 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:26.745 INFO:teuthology.orchestra.run.smithi001.stdout:1305 trunc from 0xe231b8d to 0xd684189 2024-09-13T04:58:26.756 INFO:teuthology.orchestra.run.smithi001.stdout:1306 write 0xa5d903c thru 0xa5e8fc4 (0xff89 bytes) 2024-09-13T04:58:26.761 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:26.761 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:26.762 INFO:teuthology.orchestra.run.smithi001.stdout:1307 write 0xbaa32d0 thru 0xbaad463 (0xa194 bytes) 2024-09-13T04:58:26.768 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:26.779 INFO:teuthology.orchestra.run.smithi001.stdout:1310 trunc from 0xd684189 to 0x3ff1554 2024-09-13T04:58:26.779 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:26.779 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:26.780 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:26.780 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:26.798 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:26.798 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:26.795+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:26.802 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:26.802 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:26.808 INFO:teuthology.orchestra.run.smithi001.stdout:1311 trunc from 0x3ff1554 to 0x13cd2b1 2024-09-13T04:58:26.822 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:26.822 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:26.823 INFO:teuthology.orchestra.run.smithi001.stdout:1312 read 0x9ce2d1 thru 0x9dd2ae (0xefde bytes) 2024-09-13T04:58:26.823 INFO:teuthology.orchestra.run.smithi001.stdout:1316 clone 8 order 18 su 131072 sc 13 2024-09-13T04:58:26.899 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:26.896+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:26.963 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:26.963 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:26.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:27.024 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:27.023+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:27.097 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:27.098 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:27.100 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:27.099+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:27.112 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:27.204 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:27.203+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:27.277 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:27.277 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:27.291 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:27.302 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:27.300+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:27.403 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:27.400+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:27.454 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:27.454 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:27.468 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:27.504 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:27.500+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:27.589 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:27.590 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:27.604 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:27.605 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:27.603+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:27.707 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:27.704+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:27.724 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:27.724 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:58:27.741 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:27.807 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:27.803+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:27.869 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:27.869 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:27.884 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:27.887 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-13T04:58:27.928 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:27.927+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:28.010 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:28.008+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:28.054 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:28.054 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:28.069 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:28.112 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:28.111+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:28.192 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:28.192 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:28.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:28.212 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:28.208+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:28.313 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:28.312+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:28.371 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:28.372 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:28.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:1317 trunc from 0x13cd2b1 to 0x95abd35 2024-09-13T04:58:28.414 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:28.412+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:28.425 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:28.440 INFO:teuthology.orchestra.run.smithi001.stdout:1319 writesame 0xe48752f thru 0xe494295 data_size 0x1(0xcd67 bytes) 2024-09-13T04:58:28.440 INFO:teuthology.orchestra.run.smithi001.stdout:1321 write 0xd039016 thru 0xd03dcbe (0x4ca9 bytes) 2024-09-13T04:58:28.444 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:28.445 INFO:teuthology.orchestra.run.smithi001.stdout:1323 punch from 0x4e43bba to 0x4e4f6a1, (0xbae7 bytes) 2024-09-13T04:58:28.447 INFO:teuthology.orchestra.run.smithi001.stdout:1324 trunc from 0xe494296 to 0xc5e8896 2024-09-13T04:58:28.465 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:28.465 INFO:teuthology.orchestra.run.smithi001.stdout:1327 clone 9 order 22 su 16384 sc 8 2024-09-13T04:58:28.514 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:28.515 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:28.515 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:28.511+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:28.529 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:28.616 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:28.615+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:28.701 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:28.701 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:28.716 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:28.717 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:28.716+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:28.818 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:28.815+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:28.838 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:28.851 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:28.851 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:28.854 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone8 from 0xc5e8896 (overlap 0x13cd2b1) to 0x11d2405 2024-09-13T04:58:28.869 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:28.919 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:28.916+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:28.992 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-13T04:58:28.998 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:29.018 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:29.019 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:29.015+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:1328 write 0x4662812 thru 0x4662d11 (0x500 bytes) 2024-09-13T04:58:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:29.044 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:1329 write 0x29dd65b thru 0x29ea76a (0xd110 bytes) 2024-09-13T04:58:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:1330 read 0xb054115 thru 0xb0554f3 (0x13df bytes) 2024-09-13T04:58:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:1331 punch from 0x899c138 to 0x89a1287, (0x514f bytes) 2024-09-13T04:58:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:1333 trunc from 0xc5e8896 to 0xef5835 2024-09-13T04:58:29.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:29.120 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:29.116+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:29.176 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:29.177 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:29.212 INFO:teuthology.orchestra.run.smithi001.stdout:1334 trunc from 0xef5835 to 0xdd2feb 2024-09-13T04:58:29.218 INFO:teuthology.orchestra.run.smithi001.stdout:1335 read 0xc1df2 thru 0xc733b (0x554a bytes) 2024-09-13T04:58:29.220 INFO:teuthology.orchestra.run.smithi001.stdout:1336 punch from 0xa57365 to 0xa60cfd, (0x9998 bytes) 2024-09-13T04:58:29.221 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:29.220+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:29.225 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:29.235 INFO:teuthology.orchestra.run.smithi001.stdout:1340 trunc from 0xdd2feb to 0xacb4b5f 2024-09-13T04:58:29.240 INFO:teuthology.orchestra.run.smithi001.stdout:1341 read 0x4c87880 thru 0x4c93732 (0xbeb3 bytes) 2024-09-13T04:58:29.241 INFO:teuthology.orchestra.run.smithi001.stdout:1343 writesame 0x8b6ab55 thru 0x8b74f6c data_size 0x76(0xa418 bytes) 2024-09-13T04:58:29.248 INFO:teuthology.orchestra.run.smithi001.stdout:1346 read 0x175da54 thru 0x17655aa (0x7b57 bytes) 2024-09-13T04:58:29.249 INFO:teuthology.orchestra.run.smithi001.stdout:1347 read 0x1f1a756 thru 0x1f22d7c (0x8627 bytes) 2024-09-13T04:58:29.249 INFO:teuthology.orchestra.run.smithi001.stdout:1348 read 0x88bf6ef thru 0x88c36e2 (0x3ff4 bytes) 2024-09-13T04:58:29.249 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:29.249 INFO:teuthology.orchestra.run.smithi001.stdout:1350 read 0x4dad3ac thru 0x4db33aa (0x5fff bytes) 2024-09-13T04:58:29.252 INFO:teuthology.orchestra.run.smithi001.stdout:1351 trunc from 0xacb4b5f to 0xcfef6a0 2024-09-13T04:58:29.259 INFO:teuthology.orchestra.run.smithi001.stdout:1352 read 0xc0c373f thru 0xc0d09bb (0xd27d bytes) 2024-09-13T04:58:29.259 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:29.259 INFO:teuthology.orchestra.run.smithi001.stdout:1354 writesame 0x62b80c4 thru 0x62c694a data_size 0x13(0xe887 bytes) 2024-09-13T04:58:29.269 INFO:teuthology.orchestra.run.smithi001.stdout:1357 punch from 0x113798 to 0x122685, (0xeeed bytes) 2024-09-13T04:58:29.269 INFO:teuthology.orchestra.run.smithi001.stdout:1358 read 0x27bcdaf thru 0x27c6a97 (0x9ce9 bytes) 2024-09-13T04:58:29.269 INFO:teuthology.orchestra.run.smithi001.stdout:1361 write 0x2b7be69 thru 0x2b84e72 (0x900a bytes) 2024-09-13T04:58:29.270 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:29.271 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:29.278 INFO:teuthology.orchestra.run.smithi001.stdout:1362 read 0x691df24 thru 0x6921f38 (0x4015 bytes) 2024-09-13T04:58:29.288 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:29.289 INFO:teuthology.orchestra.run.smithi001.stdout:1363 write 0xeb25b17 thru 0xeb2e9cb (0x8eb5 bytes) 2024-09-13T04:58:29.296 INFO:teuthology.orchestra.run.smithi001.stdout:1364 punch from 0x1b04a4e to 0x1b0e3df, (0x9991 bytes) 2024-09-13T04:58:29.296 INFO:teuthology.orchestra.run.smithi001.stdout:1367 read 0xe91afbb thru 0xe91fdcf (0x4e15 bytes) 2024-09-13T04:58:29.296 INFO:teuthology.orchestra.run.smithi001.stdout:1369 write 0x1cdbf9f thru 0x1cdedf1 (0x2e53 bytes) 2024-09-13T04:58:29.298 INFO:teuthology.orchestra.run.smithi001.stdout:1371 read 0xb840273 thru 0xb846e08 (0x6b96 bytes) 2024-09-13T04:58:29.305 INFO:teuthology.orchestra.run.smithi001.stdout:1372 trunc from 0xeb2e9cc to 0x9f2c3aa 2024-09-13T04:58:29.321 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:29.319+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:29.325 INFO:teuthology.orchestra.run.smithi001.stdout:1373 trunc from 0x9f2c3aa to 0xcbbc31a 2024-09-13T04:58:29.331 INFO:teuthology.orchestra.run.smithi001.stdout:1374 writesame 0x5847262 thru 0x5853093 data_size 0x2d(0xbe32 bytes) 2024-09-13T04:58:29.348 INFO:teuthology.orchestra.run.smithi001.stdout:1375 write 0xdfc8ad6 thru 0xdfd1c97 (0x91c2 bytes) 2024-09-13T04:58:29.359 INFO:teuthology.orchestra.run.smithi001.stdout:1377 write 0xdf8169 thru 0xdfa187 (0x201f bytes) 2024-09-13T04:58:29.361 INFO:teuthology.orchestra.run.smithi001.stdout:1378 read 0x7f0f573 thru 0x7f1dcea (0xe778 bytes) 2024-09-13T04:58:29.362 INFO:teuthology.orchestra.run.smithi001.stdout:1379 read 0xdcc6d35 thru 0xdccbc44 (0x4f10 bytes) 2024-09-13T04:58:29.362 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:29.363 INFO:teuthology.orchestra.run.smithi001.stdout:1381 writesame 0xabd0b5d thru 0xabd4495 data_size 0x13(0x3939 bytes) 2024-09-13T04:58:29.369 INFO:teuthology.orchestra.run.smithi001.stdout:1382 read 0x18dd1ac thru 0x18e6bf7 (0x9a4c bytes) 2024-09-13T04:58:29.369 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:29.378 INFO:teuthology.orchestra.run.smithi001.stdout:1384 trunc from 0xdfd1c98 to 0x7d4eeb6 2024-09-13T04:58:29.396 INFO:teuthology.orchestra.run.smithi001.stdout:1386 clone 10 order 24 su 8388608 sc 11 2024-09-13T04:58:29.421 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16592 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.383498253000001, 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055701260000000002, 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16592 2024-09-13T04:58:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036513+0000", 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5299999999999999e-06 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036521+0000", 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6699999999999994e-06 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036525+0000", 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3610000000000003e-06 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092051+0000", 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05552613 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092163+0000", 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111494 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092212+0000", 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9432999999999999e-05 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16593 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.383479733, 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055799934000000002, 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16593 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036532+0000", 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3810000000000002e-06 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036539+0000", 2024-09-13T04:58:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2139999999999999e-06 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036544+0000", 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1909999999999999e-06 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092237+0000", 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692609999999997 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092290+0000", 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3554000000000003e-05 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092329+0000", 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9335999999999997e-05 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16594 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.383460759, 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055959726000000001, 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16594 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036550+0000", 2024-09-13T04:58:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036557+0000", 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999999e-06 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036562+0000", 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092364+0000", 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801515000000003 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092441+0000", 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7044000000000003e-05 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092508+0000", 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7617e-05 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16595 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.383109051, 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055757809999999998, 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16595 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036566+0000", 2024-09-13T04:58:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996657 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036905+0000", 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033883999999999997 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036915+0000", 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0164e-05 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036930+0000", 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5245999999999999e-05 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092559+0000", 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055628628999999999 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092615+0000", 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6684000000000001e-05 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092658+0000", 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2559999999999999e-05 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16596 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3215360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.382871406, 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696469999999998, 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16596 2024-09-13T04:58:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036936+0000", 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997983 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037142+0000", 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020614900000000001 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037152+0000", 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6739999999999992e-06 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037159+0000", 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092692+0000", 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055533569999999997 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092768+0000", 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5220000000000001e-05 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092834+0000", 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6674999999999994e-05 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16597 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.382718544999999, 2024-09-13T04:58:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055678257000000002, 2024-09-13T04:58:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16597 2024-09-13T04:58:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037164+0000", 2024-09-13T04:58:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998736 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037300+0000", 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000135456 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037310+0000", 2024-09-13T04:58:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0366e-05 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037316+0000", 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4910000000000002e-06 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092888+0000", 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055571872000000001 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092929+0000", 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0667999999999998e-05 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092969+0000", 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9913000000000002e-05 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16603 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.381690171000001, 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055630449999999998, 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16603 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038321+0000", 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1179999999999999e-06 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038326+0000", 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6189999999999999e-06 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038331+0000", 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2429999999999999e-06 2024-09-13T04:58:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093860+0000", 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055528871 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093900+0000", 2024-09-13T04:58:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0742999999999999e-05 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093949+0000", 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16604 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.381674542000001, 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055794816999999997, 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16604 2024-09-13T04:58:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038334+0000", 2024-09-13T04:58:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038336+0000", 2024-09-13T04:58:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.046e-06 2024-09-13T04:58:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038342+0000", 2024-09-13T04:58:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038346+0000", 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1799999999999998e-06 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093983+0000", 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055636906999999999 2024-09-13T04:58:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094059+0000", 2024-09-13T04:58:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6075999999999995e-05 2024-09-13T04:58:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094129+0000", 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0337000000000001e-05 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16606 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.381165729999999, 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055591046999999998, 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16606 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038845+0000", 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4329999999999998e-06 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038852+0000", 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1240000000000002e-06 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038857+0000", 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9620000000000001e-06 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094307+0000", 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055450714999999998 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094392+0000", 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4586000000000006e-05 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094435+0000", 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2697000000000001e-05 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16607 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.381148592000001, 2024-09-13T04:58:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696915999999999, 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16607 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038860+0000", 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038862+0000", 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1859999999999999e-06 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038868+0000", 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8590000000000001e-06 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038873+0000", 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6310000000000002e-06 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094459+0000", 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055585561999999998 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094519+0000", 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0161999999999998e-05 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094558+0000", 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8831999999999997e-05 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16608 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.381132333, 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801370000000003, 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16608 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038879+0000", 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.21e-06 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038885+0000", 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8100000000000003e-06 2024-09-13T04:58:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038889+0000", 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2760000000000002e-06 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094581+0000", 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692236999999999 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094639+0000", 2024-09-13T04:58:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7935e-05 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094678+0000", 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9267999999999997e-05 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16609 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.380902456999999, 2024-09-13T04:58:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055693234000000001, 2024-09-13T04:58:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16609 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038892+0000", 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997854 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039112+0000", 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000219402 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039125+0000", 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039135+0000", 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8549999999999997e-06 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094702+0000", 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055567083000000003 2024-09-13T04:58:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094761+0000", 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9425000000000001e-05 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094800+0000", 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8754999999999997e-05 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16610 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.380817349999999, 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0557396, 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16610 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039158+0000", 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999666 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039196+0000", 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7551e-05 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039212+0000", 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6569999999999999e-05 2024-09-13T04:58:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039219+0000", 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5080000000000002e-06 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094824+0000", 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055604690999999998 2024-09-13T04:58:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094890+0000", 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6204999999999994e-05 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094931+0000", 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1675000000000003e-05 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16636 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.377108541, 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055573821000000002, 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16636 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042900+0000", 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042902+0000", 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0590000000000001e-06 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042908+0000", 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5620000000000003e-06 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042912+0000", 2024-09-13T04:58:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.437e-06 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098321+0000", 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055408959000000001 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098401+0000", 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0122000000000003e-05 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098475+0000", 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3002000000000003e-05 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16638 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.376777091999999, 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055689097, 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16638 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043234+0000", 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043237+0000", 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6260000000000001e-06 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043240+0000", 2024-09-13T04:58:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.627e-06 2024-09-13T04:58:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098773+0000", 2024-09-13T04:58:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055532519000000002 2024-09-13T04:58:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098849+0000", 2024-09-13T04:58:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6808000000000001e-05 2024-09-13T04:58:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098921+0000", 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1835000000000006e-05 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16683 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.368726196000001, 2024-09-13T04:58:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055570913999999999, 2024-09-13T04:58:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16683 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051285+0000", 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051294+0000", 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2239999999999995e-06 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051299+0000", 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0259999999999998e-06 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106697+0000", 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055398672000000003 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106781+0000", 2024-09-13T04:58:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3849000000000003e-05 2024-09-13T04:58:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106854+0000", 2024-09-13T04:58:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2683999999999997e-05 2024-09-13T04:58:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16684 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.368705733000001, 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055767156999999998, 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16684 2024-09-13T04:58:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051305+0000", 2024-09-13T04:58:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.204e-06 2024-09-13T04:58:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051313+0000", 2024-09-13T04:58:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4830000000000002e-06 2024-09-13T04:58:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051318+0000", 2024-09-13T04:58:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6529999999999999e-06 2024-09-13T04:58:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106896+0000", 2024-09-13T04:58:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055577731999999998 2024-09-13T04:58:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106987+0000", 2024-09-13T04:58:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0920000000000004e-05 2024-09-13T04:58:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107071+0000", 2024-09-13T04:58:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3492999999999995e-05 2024-09-13T04:58:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16685 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.368686584000001, 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055971087000000003, 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16685 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051322+0000", 2024-09-13T04:58:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051325+0000", 2024-09-13T04:58:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4499999999999998e-06 2024-09-13T04:58:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051332+0000", 2024-09-13T04:58:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2409999999999998e-06 2024-09-13T04:58:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051337+0000", 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0499999999999999e-06 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107129+0000", 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055791855000000001 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107212+0000", 2024-09-13T04:58:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3046000000000004e-05 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107294+0000", 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1773000000000005e-05 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16689 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.367422319999999, 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694117000000001, 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16689 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052589+0000", 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4420000000000001e-06 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052595+0000", 2024-09-13T04:58:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8549999999999998e-06 2024-09-13T04:58:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052600+0000", 2024-09-13T04:58:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8169999999999998e-06 2024-09-13T04:58:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108111+0000", 2024-09-13T04:58:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055511660999999997 2024-09-13T04:58:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108201+0000", 2024-09-13T04:58:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9386000000000001e-05 2024-09-13T04:58:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108281+0000", 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0338000000000002e-05 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16691 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.367006076999999, 2024-09-13T04:58:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055651254999999997, 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16691 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053005+0000", 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4490000000000002e-06 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053010+0000", 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2680000000000002e-06 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053014+0000", 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9729999999999999e-06 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108498+0000", 2024-09-13T04:58:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055483769000000002 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108580+0000", 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1638999999999995e-05 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108654+0000", 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4548000000000005e-05 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:29.538 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:29.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:29.540 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:29.535+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:29.640 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:29.639+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:29.711 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:29.711 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:29.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:29.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:58:29.777 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:29.776+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:29.832 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone9 from 0x7d4eeb6 (overlap 0xdd2feb) to 0x0 2024-09-13T04:58:29.844 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:29.840+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:29.861 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:29.861 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:29.861 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:29.861 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:29.862 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:29.877 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:29.943 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-13T04:58:29.952 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:29.948+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:29.986 INFO:teuthology.orchestra.run.smithi001.stdout:1387 trunc from 0x7d4eeb6 to 0x5965a9a 2024-09-13T04:58:30.046 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:30.043+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:30.061 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:30.061 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:30.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.598316240999999, 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:58:30.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:37637 3.11 e41/24 3:88b77d11:::rbd_data.112a66af081a.0000000000000000:head v 41'72, mlcod=41'70)", 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.7041172849999997, 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027255564999999999, 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355356+0000", 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356532+0000", 2024-09-13T04:58:30.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001176599 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356563+0000", 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0916e-05 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356576+0000", 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3293e-05 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356700+0000", 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123779 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356746+0000", 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6242000000000002e-05 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382519+0000", 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025772856 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382614+0000", 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4315999999999996e-05 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39466 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.96316986699999996, 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022036103000000001, 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39466 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096315+0000", 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-13T04:58:30.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096326+0000", 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1596999999999999e-05 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096332+0000", 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8560000000000003e-06 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112144+0000", 2024-09-13T04:58:30.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015812001999999999 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112220+0000", 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5354999999999999e-05 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.118342+0000", 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061219409999999997 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39993 3.16 3:6fe11eb9:::rbd_data.112a66af081a.0000000000000001:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131185+0000", 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92829004199999998, 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019162558, 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39993 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.076 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131185+0000", 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131185+0000", 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131185+0000", 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131187+0000", 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0040000000000002e-06 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131195+0000", 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8650000000000001e-06 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131207+0000", 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1977999999999999e-05 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150180+0000", 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018973515999999999 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150264+0000", 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3455000000000006e-05 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150348+0000", 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4067000000000007e-05 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39999 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 376832~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.080 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92824043599999995, 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019234370000000001, 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39999 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131234+0000", 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131237+0000", 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0839999999999999e-06 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131253+0000", 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6920999999999999e-05 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131259+0000", 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.716e-06 2024-09-13T04:58:30.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150375+0000", 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019115732999999999 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150416+0000", 2024-09-13T04:58:30.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0791999999999998e-05 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150469+0000", 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519e-05 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40000 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 507904~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92821114100000002, 2024-09-13T04:58:30.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019326534999999999, 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40000 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:30.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131271+0000", 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9330000000000004e-06 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131278+0000", 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9269999999999999e-06 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131282+0000", 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8729999999999998e-06 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150497+0000", 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019214709 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150537+0000", 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0197000000000002e-05 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150591+0000", 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3352e-05 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40001 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 638976~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:30.091 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92818805599999998, 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019422915999999998, 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40001 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131289+0000", 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.491e-06 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131306+0000", 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6489000000000001e-05 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131311+0000", 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150619+0000", 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019307642 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150657+0000", 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8015000000000001e-05 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150710+0000", 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3109999999999998e-05 2024-09-13T04:58:30.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40004 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 770048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92815929399999997, 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019514348000000001, 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40004 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131318+0000", 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8159999999999998e-06 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131325+0000", 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5370000000000003e-06 2024-09-13T04:58:30.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131330+0000", 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6619999999999996e-06 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150741+0000", 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019410981000000001 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150778+0000", 2024-09-13T04:58:30.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7793999999999998e-05 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150830+0000", 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1974000000000001e-05 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40006 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 901120~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:30.100 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92814137900000004, 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019616892, 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40006 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:30.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131336+0000", 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1940000000000001e-06 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131342+0000", 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0599999999999996e-06 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131346+0000", 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5569999999999997e-06 2024-09-13T04:58:30.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150858+0000", 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019512057999999999 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150897+0000", 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8903999999999999e-05 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150951+0000", 2024-09-13T04:58:30.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3467999999999997e-05 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40008 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1032192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92812472700000004, 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019721992000000001, 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40008 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131350+0000", 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131360+0000", 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5079999999999994e-06 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131367+0000", 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8210000000000004e-06 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131371+0000", 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8860000000000003e-06 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150979+0000", 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019607655000000002 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151020+0000", 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0921999999999999e-05 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151073+0000", 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2567e-05 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40010 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1163264~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.928099122, 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019817818000000001, 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40010 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:30.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:30.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131378+0000", 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:58:30.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131384+0000", 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0780000000000001e-06 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131389+0000", 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5079999999999999e-06 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151101+0000", 2024-09-13T04:58:30.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019712159 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151141+0000", 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0679000000000001e-05 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151194+0000", 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2589000000000001e-05 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40012 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1294336~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92808276499999998, 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019922019999999999, 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40012 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131392+0000", 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131394+0000", 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.029e-06 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131400+0000", 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0560000000000001e-06 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131424+0000", 2024-09-13T04:58:30.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3288000000000001e-05 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151223+0000", 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019799231 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151263+0000", 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9920999999999999e-05 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151315+0000", 2024-09-13T04:58:30.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1826000000000002e-05 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40014 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1425408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.928046229, 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020004087, 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40014 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131432+0000", 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9780000000000001e-06 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131441+0000", 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9590000000000001e-06 2024-09-13T04:58:30.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131447+0000", 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9410000000000004e-06 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151343+0000", 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019896113999999999 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151381+0000", 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7969999999999997e-05 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151433+0000", 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2543000000000002e-05 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40016 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1556480~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:30.123 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92799379199999998, 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020070607000000001, 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40016 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:30.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131481+0000", 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131484+0000", 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.326e-06 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131492+0000", 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0139999999999998e-06 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131505+0000", 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151461+0000", 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01995535 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151500+0000", 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8760999999999997e-05 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151552+0000", 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2533e-05 2024-09-13T04:58:30.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40018 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1687552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92796396800000003, 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020160419999999998, 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40018 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131514+0000", 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0740000000000001e-06 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131520+0000", 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4479999999999999e-06 2024-09-13T04:58:30.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131526+0000", 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151580+0000", 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020054424000000001 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151619+0000", 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9229e-05 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151672+0000", 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2493e-05 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40019 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 114688~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92794499500000005, 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020264445999999998, 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40019 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.133 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131532+0000", 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.367e-06 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131538+0000", 2024-09-13T04:58:30.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1070000000000001e-06 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131543+0000", 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3930000000000001e-06 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151700+0000", 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020157191000000001 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151741+0000", 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0864e-05 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151795+0000", 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3896e-05 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40020 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92792809399999998, 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020368440000000002, 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40020 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:30.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131549+0000", 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1110000000000002e-06 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131555+0000", 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3350000000000003e-06 2024-09-13T04:58:30.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131560+0000", 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151823+0000", 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020262782999999999 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151863+0000", 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0151000000000002e-05 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151916+0000", 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2604e-05 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.143 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40021 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1949696~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92791123499999995, 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020470662000000001, 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40021 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131566+0000", 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.001e-06 2024-09-13T04:58:30.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131571+0000", 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5110000000000003e-06 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131576+0000", 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151943+0000", 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020366649000000001 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151981+0000", 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8756e-05 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152035+0000", 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3335999999999999e-05 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.148 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40023 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92788880200000001, 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020567854, 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40023 2024-09-13T04:58:30.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:30.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131589+0000", 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4140000000000001e-06 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131597+0000", 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5220000000000007e-06 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131613+0000", 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5628999999999999e-05 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152063+0000", 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020450032 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152101+0000", 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7886999999999998e-05 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152154+0000", 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3739999999999997e-05 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40052 3.e 3:75db92e1:::rbd_header.112a6578f5c6:head [watch reconnect cookie 139644891391936 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.92785685299999998, 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021419572000000001, 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.154 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.154 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.154 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40052 2024-09-13T04:58:30.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.154 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131621+0000", 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9280000000000002e-06 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131635+0000", 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4467000000000001e-05 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131642+0000", 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.144e-06 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152171+0000", 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020529544 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152220+0000", 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.153038+0000", 2024-09-13T04:58:30.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00081775299999999995 2024-09-13T04:58:30.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.158 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.158 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.158 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:30.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:30.160 INFO:teuthology.orchestra.run.smithi001.stdout:1388 read 0x1729ffa thru 0x1739cd0 (0xfcd7 bytes) 2024-09-13T04:58:30.161 INFO:teuthology.orchestra.run.smithi001.stdout:1389 write 0x3cda10 thru 0x3d7e6e (0xa45f bytes) 2024-09-13T04:58:30.161 INFO:teuthology.orchestra.run.smithi001.stdout:1390 writesame 0xd2820e9 thru 0xd291306 data_size 0x11(0xf21e bytes) 2024-09-13T04:58:30.161 INFO:teuthology.orchestra.run.smithi001.stdout:1392 write 0x920a027 thru 0x92178bc (0xd896 bytes) 2024-09-13T04:58:30.161 INFO:teuthology.orchestra.run.smithi001.stdout:1393 write 0x3020833 thru 0x302479c (0x3f6a bytes) 2024-09-13T04:58:30.161 INFO:teuthology.orchestra.run.smithi001.stdout:1394 write 0x8fbf5d7 thru 0x8fc3fee (0x4a18 bytes) 2024-09-13T04:58:30.161 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:30.156+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:30.169 INFO:teuthology.orchestra.run.smithi001.stdout:1395 write 0xe574525 thru 0xe577314 (0x2df0 bytes) 2024-09-13T04:58:30.174 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.174 INFO:teuthology.orchestra.run.smithi001.stdout:1399 punch from 0x90973bc to 0x90a2aa9, (0xb6ed bytes) 2024-09-13T04:58:30.175 INFO:teuthology.orchestra.run.smithi001.stdout:1401 read 0x5e301c4 thru 0x5e38e61 (0x8c9e bytes) 2024-09-13T04:58:30.175 INFO:teuthology.orchestra.run.smithi001.stdout:1402 read 0xc31973b thru 0xc323a67 (0xa32d bytes) 2024-09-13T04:58:30.175 INFO:teuthology.orchestra.run.smithi001.stdout:1404 write 0x8366e3b thru 0x8368243 (0x1409 bytes) 2024-09-13T04:58:30.185 INFO:teuthology.orchestra.run.smithi001.stdout:1405 read 0x28e0193 thru 0x28e2b32 (0x29a0 bytes) 2024-09-13T04:58:30.186 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.186 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.198 INFO:teuthology.orchestra.run.smithi001.stdout:1409 writesame 0xedb7255 thru 0xedbd0c4 data_size 0x8(0x5e70 bytes) 2024-09-13T04:58:30.198 INFO:teuthology.orchestra.run.smithi001.stdout:1410 read 0x33d4451 thru 0x33dd025 (0x8bd5 bytes) 2024-09-13T04:58:30.199 INFO:teuthology.orchestra.run.smithi001.stdout:1411 punch from 0xea97de3 to 0xeaa1058, (0x9275 bytes) 2024-09-13T04:58:30.199 INFO:teuthology.orchestra.run.smithi001.stdout:1413 writesame 0x33a1a79 thru 0x33a6e37 data_size 0x1(0x53bf bytes) 2024-09-13T04:58:30.216 INFO:teuthology.orchestra.run.smithi001.stdout:1414 write 0x1311a07 thru 0x1318f3e (0x7538 bytes) 2024-09-13T04:58:30.221 INFO:teuthology.orchestra.run.smithi001.stdout:1415 read 0x396c3e7 thru 0x396ee46 (0x2a60 bytes) 2024-09-13T04:58:30.221 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.232 INFO:teuthology.orchestra.run.smithi001.stdout:1417 trunc from 0xedbd0c5 to 0x8587078 2024-09-13T04:58:30.260 INFO:teuthology.orchestra.run.smithi001.stdout:1418 write 0x82b3c03 thru 0x82bfee3 (0xc2e1 bytes) 2024-09-13T04:58:30.260 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:30.256+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:30.269 INFO:teuthology.orchestra.run.smithi001.stdout:1419 trunc from 0x8587078 to 0x937eeb3 2024-09-13T04:58:30.281 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.298 INFO:teuthology.orchestra.run.smithi001.stdout:1422 writesame 0xb6786cb thru 0xb681598 data_size 0x1(0x8ece bytes) 2024-09-13T04:58:30.300 INFO:teuthology.orchestra.run.smithi001.stdout:1423 trunc from 0xb681599 to 0xa033af6 2024-09-13T04:58:30.311 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:30.311 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:30.311 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:30.311 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:30.311 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.317 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:58:30.317 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:58:30.317 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T04:58:30.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T04:58:30.331 INFO:teuthology.orchestra.run.smithi001.stdout:1427 trunc from 0xa033af6 to 0x51fa55 2024-09-13T04:58:30.334 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:30.362 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:30.359+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:30.377 INFO:teuthology.orchestra.run.smithi001.stdout:1428 trunc from 0x51fa55 to 0x6a8bded 2024-09-13T04:58:30.395 INFO:teuthology.orchestra.run.smithi001.stdout:1429 trunc from 0x6a8bded to 0xa962209 2024-09-13T04:58:30.405 INFO:teuthology.orchestra.run.smithi001.stdout:1432 read 0x9d9fa2f thru 0x9dac573 (0xcb45 bytes) 2024-09-13T04:58:30.405 INFO:teuthology.orchestra.run.smithi001.stdout:1433 writesame 0x3e186f3 thru 0x3e1db04 data_size 0x33(0x5412 bytes) 2024-09-13T04:58:30.420 INFO:teuthology.orchestra.run.smithi001.stdout:1434 writesame 0xb920592 thru 0xb92305c data_size 0x7(0x2acb bytes) 2024-09-13T04:58:30.421 INFO:teuthology.orchestra.run.smithi001.stdout:1436 read 0xa622b3b thru 0xa62d91b (0xade1 bytes) 2024-09-13T04:58:30.432 INFO:teuthology.orchestra.run.smithi001.stdout:1437 write 0xcd8bb67 thru 0xcd98ddf (0xd279 bytes) 2024-09-13T04:58:30.437 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.437 INFO:teuthology.orchestra.run.smithi001.stdout:1441 writesame 0xb142020 thru 0xb14d426 data_size 0x1(0xb407 bytes) 2024-09-13T04:58:30.462 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:30.459+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:30.470 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.470 INFO:teuthology.orchestra.run.smithi001.stdout:1444 write 0x92cbf41 thru 0x92d2585 (0x6645 bytes) 2024-09-13T04:58:30.475 INFO:teuthology.orchestra.run.smithi001.stdout:1445 trunc from 0xcd98de0 to 0xd757d6e 2024-09-13T04:58:30.485 INFO:teuthology.orchestra.run.smithi001.stdout:1446 writesame 0xfd5cfb thru 0xfda635 data_size 0x3(0x493b bytes) 2024-09-13T04:58:30.494 INFO:teuthology.orchestra.run.smithi001.stdout:1447 trunc from 0xd757d6e to 0xe0c10ac 2024-09-13T04:58:30.503 INFO:teuthology.orchestra.run.smithi001.stdout:1448 read 0x5d4dce8 thru 0x5d522f6 (0x460f bytes) 2024-09-13T04:58:30.504 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.504 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.504 INFO:teuthology.orchestra.run.smithi001.stdout:1451 write 0x9d5590 thru 0x9e2969 (0xd3da bytes) 2024-09-13T04:58:30.509 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:30.509 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:30.509 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:30.509 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:30.509 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:30.512 INFO:teuthology.orchestra.run.smithi001.stdout:1452 read 0x8744350 thru 0x874b5c7 (0x7278 bytes) 2024-09-13T04:58:30.520 INFO:teuthology.orchestra.run.smithi001.stdout:1454 trunc from 0xe0c10ac to 0xa06d124 2024-09-13T04:58:30.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:30.551 INFO:teuthology.orchestra.run.smithi001.stdout:1455 write 0xc62b1f3 thru 0xc6382a9 (0xd0b7 bytes) 2024-09-13T04:58:30.555 INFO:teuthology.orchestra.run.smithi001.stdout:1456 write 0x29f323c thru 0x29fe7b5 (0xb57a bytes) 2024-09-13T04:58:30.563 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:30.560+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:30.565 INFO:teuthology.orchestra.run.smithi001.stdout:1457 read 0xc4d3df4 thru 0xc4d7c02 (0x3e0f bytes) 2024-09-13T04:58:30.565 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.580 INFO:teuthology.orchestra.run.smithi001.stdout:1459 trunc from 0xc6382aa to 0x26ce75f 2024-09-13T04:58:30.601 INFO:teuthology.orchestra.run.smithi001.stdout:1461 write 0x235d851 thru 0x236c868 (0xf018 bytes) 2024-09-13T04:58:30.606 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.618 INFO:teuthology.orchestra.run.smithi001.stdout:1466 write 0x42ce55d thru 0x42dc08d (0xdb31 bytes) 2024-09-13T04:58:30.623 INFO:teuthology.orchestra.run.smithi001.stdout:1467 punch from 0x372a42e to 0x3738eda, (0xeaac bytes) 2024-09-13T04:58:30.623 INFO:teuthology.orchestra.run.smithi001.stdout:1469 read 0x332e221 thru 0x3338d29 (0xab09 bytes) 2024-09-13T04:58:30.650 INFO:teuthology.orchestra.run.smithi001.stdout:1470 write 0xe7fe660 thru 0xe8095b4 (0xaf55 bytes) 2024-09-13T04:58:30.657 INFO:teuthology.orchestra.run.smithi001.stdout:1471 write 0xe527edb thru 0xe528dc7 (0xeed bytes) 2024-09-13T04:58:30.663 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:30.660+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:30.664 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 44, 2024-09-13T04:58:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "8850da77-4384-4335-9559-af2e805c4f72", 2024-09-13T04:58:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-13T04:57:34.164121+0000", 2024-09-13T04:58:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-13T04:58:29.726687+0000", 2024-09-13T04:58:30.664 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-13T04:57:47.098606+0000", 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-13T04:57:36.815703+0000", 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5801984, 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "noscrub", 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T04:58:30.665 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "reef", 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-13T04:58:30.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:48.984599+0000", 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:30.667 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:30.668 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "19", 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:30.669 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:30.670 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:30.671 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:30.672 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.673 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-13T04:58:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-13T04:58:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:52.756781+0000", 2024-09-13T04:58:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:58:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:58:30.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:30.675 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-13T04:58:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-13T04:58:30.676 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:30.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "23", 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 3, 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 23, 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:30.678 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:30.679 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.680 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:30.681 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:58:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:58:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.682 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 4.9800000190734863, 2024-09-13T04:58:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 4.9800000190734863, 2024-09-13T04:58:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T04:58:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 4.380000114440918, 2024-09-13T04:58:30.683 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 4.380000114440918, 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:58:02.446076+0000", 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:58:30.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-13T04:58:30.685 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "44", 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:30.686 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 13, 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 44, 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:30.687 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:30.688 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:30.689 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-13T04:58:30.690 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "2fda29ca-3c6d-47ae-9244-8ccdf55f89f6", 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:30.691 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:30.692 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:30.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:30.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-13T04:58:30.695 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6820/2346968854", 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6824/2346968854", 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/2346968854", 2024-09-13T04:58:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/2346968854", 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9ca6e867-faf0-4295-b8eb-4c738dd26cb9", 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:30.698 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-13T04:58:30.699 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.700 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:30.702 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.703 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/466532899", 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6821/466532899", 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/466532899", 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/466532899", 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d8159400-8662-4567-8714-b48c2d8f3ad2", 2024-09-13T04:58:30.704 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:30.705 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:30.705 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:30.705 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:30.705 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:30.705 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:30.705 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:30.705 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:30.705 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:30.706 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:30.706 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:30.706 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.706 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-13T04:58:30.706 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:30.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:30.707 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.708 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-13T04:58:30.708 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:30.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.708 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-13T04:58:30.709 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:30.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.709 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:30.709 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.710 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:30.711 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.711 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-13T04:58:30.711 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:30.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.712 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-13T04:58:30.712 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:30.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/343712578", 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/343712578", 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/343712578", 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/343712578", 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:30.713 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:58:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "13a3ea70-fa37-41d8-bae7-0203952b02cb", 2024-09-13T04:58:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:30.714 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.716 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.717 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:30.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.720 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-13T04:58:30.720 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:30.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.720 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.720 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/2021827348", 2024-09-13T04:58:30.720 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/2021827348", 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/2021827348", 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/2021827348", 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:58:30.721 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e896e6b5-c54a-405f-ab69-2a5b7b7a153e", 2024-09-13T04:58:30.722 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:30.722 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:30.722 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:30.722 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:30.722 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6800", 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6801", 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6802", 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:30.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6803", 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6806", 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6807", 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6804", 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6805", 2024-09-13T04:58:30.727 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6801/2183053562", 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6803/2183053562", 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6807/2183053562", 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6805/2183053562", 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:30.728 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7de23072-7eda-43d6-9b92-d608ef82c6fa", 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:30.729 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6816", 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6817", 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6819", 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6821", 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6826", 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.733 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6828", 2024-09-13T04:58:30.733 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:30.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.733 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.734 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:30.734 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.734 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6822", 2024-09-13T04:58:30.735 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:30.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.735 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6824", 2024-09-13T04:58:30.736 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:30.736 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.736 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.736 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6817/2765053006", 2024-09-13T04:58:30.736 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6821/2765053006", 2024-09-13T04:58:30.737 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6828/2765053006", 2024-09-13T04:58:30.737 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6824/2765053006", 2024-09-13T04:58:30.737 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:30.737 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:30.737 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8719fad9-81a0-46be-a546-db83bcace33b", 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:30.738 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 36, 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6818", 2024-09-13T04:58:30.739 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6820", 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6823", 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6825", 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:30.741 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6830", 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6831", 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:30.742 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6827", 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6829", 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6820/2721913793", 2024-09-13T04:58:30.743 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6825/2721913793", 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6831/2721913793", 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6829/2721913793", 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8bf0a351-e58e-4393-a6ce-fe70acf362aa", 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:30.744 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6808", 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6809", 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:30.746 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6810", 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6811", 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.747 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6814", 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6815", 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6812", 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:30.749 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6813", 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6809/4202513770", 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6811/4202513770", 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6815/4202513770", 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6813/4202513770", 2024-09-13T04:58:30.750 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:30.751 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.092681+0000", 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:58:30.752 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:30.753 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:30.753 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:30.753 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:30.753 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:30.754 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.414226+0000", 2024-09-13T04:58:30.754 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:30.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.754 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:58:30.754 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:30.755 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:30.755 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:30.755 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:30.756 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:30.756 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.368701+0000", 2024-09-13T04:58:30.756 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:30.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.757 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:58:30.757 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:30.757 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:30.757 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:30.757 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:30.757 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:30.758 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.370707+0000", 2024-09-13T04:58:30.758 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:30.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.758 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:58:30.758 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:30.759 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:30.759 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:30.759 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:30.759 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:30.759 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.980085+0000", 2024-09-13T04:58:30.759 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:30.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.760 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:58:30.760 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:30.760 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:30.760 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:30.761 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:30.761 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:30.761 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.874193+0000", 2024-09-13T04:58:30.761 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:30.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.762 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:58:30.762 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:30.762 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:30.762 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:30.762 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:30.763 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:30.763 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.219724+0000", 2024-09-13T04:58:30.763 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:30.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.763 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:58:30.764 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:30.764 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:30.764 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:30.764 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:30.764 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:30.765 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.114987+0000", 2024-09-13T04:58:30.765 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:30.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.765 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:30.765 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [ 2024-09-13T04:58:30.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.766 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.6", 2024-09-13T04:58:30.766 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:58:30.766 INFO:teuthology.orchestra.run.smithi001.stdout: 6, 2024-09-13T04:58:30.766 INFO:teuthology.orchestra.run.smithi001.stdout: 4 2024-09-13T04:58:30.766 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.767 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:30.767 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2024-09-13T04:58:30.767 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-13T04:58:30.767 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-13T04:58:30.767 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-13T04:58:30.767 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-13T04:58:30.768 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-13T04:58:30.768 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-13T04:58:30.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-13T04:58:30.768 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-13T04:58:30.768 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-13T04:58:30.768 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-13T04:58:30.769 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-13T04:58:30.769 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-13T04:58:30.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.769 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-13T04:58:30.770 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-13T04:58:30.770 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-13T04:58:30.770 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-13T04:58:30.770 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-13T04:58:30.770 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-13T04:58:30.770 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-13T04:58:30.771 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-13T04:58:30.771 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-13T04:58:30.771 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-13T04:58:30.771 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-13T04:58:30.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.771 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:30.773 INFO:teuthology.orchestra.run.smithi001.stdout:1472 trunc from 0xe8095b5 to 0x6af96a6 2024-09-13T04:58:30.776 INFO:tasks.thrashosds.thrasher:j is {'epoch': 44, 'fsid': '8850da77-4384-4335-9559-af2e805c4f72', 'created': '2024-09-13T04:57:34.164121+0000', 'modified': '2024-09-13T04:58:29.726687+0000', 'last_up_change': '2024-09-13T04:57:47.098606+0000', 'last_in_change': '2024-09-13T04:57:36.815703+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T04:57:48.984599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T04:57:52.756781+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T04:58:02.446076+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2fda29ca-3c6d-47ae-9244-8ccdf55f89f6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6820', 'nonce': 2346968854}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6824', 'nonce': 2346968854}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 2346968854}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 2346968854}]}, 'public_addr': '172.21.15.1:6820/2346968854', 'cluster_addr': '172.21.15.1:6824/2346968854', 'heartbeat_back_addr': '172.21.15.1:6831/2346968854', 'heartbeat_front_addr': '172.21.15.1:6828/2346968854', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9ca6e867-faf0-4295-b8eb-4c738dd26cb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 466532899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6819', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 466532899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 466532899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6823', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 466532899}]}, 'public_addr': '172.21.15.1:6817/466532899', 'cluster_addr': '172.21.15.1:6821/466532899', 'heartbeat_back_addr': '172.21.15.1:6829/466532899', 'heartbeat_front_addr': '172.21.15.1:6825/466532899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8159400-8662-4567-8714-b48c2d8f3ad2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 343712578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 343712578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 343712578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 343712578}]}, 'public_addr': '172.21.15.1:6801/343712578', 'cluster_addr': '172.21.15.1:6803/343712578', 'heartbeat_back_addr': '172.21.15.1:6807/343712578', 'heartbeat_front_addr': '172.21.15.1:6805/343712578', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '13a3ea70-fa37-41d8-bae7-0203952b02cb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 2021827348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 2021827348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 2021827348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 2021827348}]}, 'public_addr': '172.21.15.1:6809/2021827348', 'cluster_addr': '172.21.15.1:6811/2021827348', 'heartbeat_back_addr': '172.21.15.1:6815/2021827348', 'heartbeat_front_addr': '172.21.15.1:6813/2021827348', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e896e6b5-c54a-405f-ab69-2a5b7b7a153e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 2183053562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 2183053562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 2183053562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 2183053562}]}, 'public_addr': '172.21.15.144:6801/2183053562', 'cluster_addr': '172.21.15.144:6803/2183053562', 'heartbeat_back_addr': '172.21.15.144:6807/2183053562', 'heartbeat_front_addr': '172.21.15.144:6805/2183053562', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7de23072-7eda-43d6-9b92-d608ef82c6fa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6816', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2765053006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 2765053006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6826', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6828', 'nonce': 2765053006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6824', 'nonce': 2765053006}]}, 'public_addr': '172.21.15.144:6817/2765053006', 'cluster_addr': '172.21.15.144:6821/2765053006', 'heartbeat_back_addr': '172.21.15.144:6828/2765053006', 'heartbeat_front_addr': '172.21.15.144:6824/2765053006', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8719fad9-81a0-46be-a546-db83bcace33b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2721913793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6823', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6825', 'nonce': 2721913793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6830', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6831', 'nonce': 2721913793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6827', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6829', 'nonce': 2721913793}]}, 'public_addr': '172.21.15.144:6820/2721913793', 'cluster_addr': '172.21.15.144:6825/2721913793', 'heartbeat_back_addr': '172.21.15.144:6831/2721913793', 'heartbeat_front_addr': '172.21.15.144:6829/2721913793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8bf0a351-e58e-4393-a6ce-fe70acf362aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6809', 'nonce': 4202513770}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6810', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 4202513770}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6815', 'nonce': 4202513770}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 4202513770}]}, 'public_addr': '172.21.15.144:6809/4202513770', 'cluster_addr': '172.21.15.144:6811/4202513770', 'heartbeat_back_addr': '172.21.15.144:6815/4202513770', 'heartbeat_front_addr': '172.21.15.144:6813/4202513770', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.092681+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.414226+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.368701+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.370707+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.980085+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.874193+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.219724+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.114987+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.6', 'osds': [6, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T04:58:30.776 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:58:30.777 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:30.777 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:30.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:30.777 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:30.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.778 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:58:30.778 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:30.778 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.524193031000003, 2024-09-13T04:58:30.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:58:30.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:30.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:58:30.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:58:30.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:58:30.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.780 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:30.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:30.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:58:30.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:58:30.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:58:30.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:58:30.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:58:30.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:15554 3.a 3:54a75321:::rbd_data.112af6c12117.0000000000000002:6 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.662824864999999, 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047745517000000001, 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 15554 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025346+0000", 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025348+0000", 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1900000000000002e-06 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025355+0000", 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7850000000000003e-06 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025360+0000", 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6789999999999996e-06 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.067347+0000", 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041987402 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.072996+0000", 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056486540000000003 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.073092+0000", 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6151999999999997e-05 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.521752372, 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17859 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1458176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:30.796 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.397384493000001, 2024-09-13T04:58:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047596271000000002, 2024-09-13T04:58:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17859 2024-09-13T04:58:30.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.798 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:30.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290789+0000", 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290794+0000", 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7610000000000004e-06 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290799+0000", 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4889999999999997e-06 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333438+0000", 2024-09-13T04:58:30.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042639200000000002 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333513+0000", 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5172000000000005e-05 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.338383+0000", 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048697929999999999 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17893 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393875080000001, 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046056773000000002, 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17893 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294298+0000", 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5000000000000002e-06 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294307+0000", 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294312+0000", 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.451e-06 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340260+0000", 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045947904999999997 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340302+0000", 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1650999999999998e-05 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340353+0000", 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1208000000000002e-05 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.807 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17894 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393854574000001, 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046155512000000003, 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17894 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.808 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294316+0000", 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294319+0000", 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2390000000000001e-06 2024-09-13T04:58:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294325+0000", 2024-09-13T04:58:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0340000000000002e-06 2024-09-13T04:58:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294329+0000", 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7509999999999999e-06 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340378+0000", 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046048248 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340421+0000", 2024-09-13T04:58:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3627000000000001e-05 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340472+0000", 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17895 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393837789999999, 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046268562999999999, 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17895 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294333+0000", 2024-09-13T04:58:30.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294335+0000", 2024-09-13T04:58:30.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-13T04:58:30.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294341+0000", 2024-09-13T04:58:30.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9939999999999997e-06 2024-09-13T04:58:30.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294346+0000", 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7500000000000003e-06 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340508+0000", 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046161354000000002 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340550+0000", 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2962e-05 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340602+0000", 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1665000000000001e-05 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17896 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1912832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393821042000001, 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046367673999999998, 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17896 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:30.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:30.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294352+0000", 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.17e-06 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294358+0000", 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2249999999999997e-06 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294363+0000", 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6990000000000002e-06 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340627+0000", 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046263836000000003 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340668+0000", 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0638e-05 2024-09-13T04:58:30.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340718+0000", 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0458999999999999e-05 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17897 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1953792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.824 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393804233000001, 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046474117000000002, 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17897 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294369+0000", 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294375+0000", 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9880000000000001e-06 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294381+0000", 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7939999999999999e-06 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340744+0000", 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046362962000000001 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340786+0000", 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2311000000000002e-05 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340841+0000", 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5192999999999998e-05 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17898 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1994752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.39378617, 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046575812000000001, 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17898 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:30.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294387+0000", 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1600000000000001e-06 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294395+0000", 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.52e-06 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294400+0000", 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3149999999999999e-06 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340868+0000", 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046467893000000003 2024-09-13T04:58:30.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340911+0000", 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3340000000000002e-05 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340961+0000", 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9934e-05 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17899 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2035712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393767511, 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691141999999998, 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17899 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.835 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294406+0000", 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294411+0000", 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294416+0000", 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5600000000000004e-06 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340987+0000", 2024-09-13T04:58:30.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046570645000000001 2024-09-13T04:58:30.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341045+0000", 2024-09-13T04:58:30.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8065000000000001e-05 2024-09-13T04:58:30.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341095+0000", 2024-09-13T04:58:30.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0302000000000001e-05 2024-09-13T04:58:30.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.841 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17900 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2076672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.841 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:30.841 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393751048, 2024-09-13T04:58:30.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046796380999999998, 2024-09-13T04:58:30.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17900 2024-09-13T04:58:30.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.842 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:30.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:30.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:30.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294422+0000", 2024-09-13T04:58:30.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9609999999999999e-06 2024-09-13T04:58:30.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294428+0000", 2024-09-13T04:58:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9290000000000003e-06 2024-09-13T04:58:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294432+0000", 2024-09-13T04:58:30.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4190000000000004e-06 2024-09-13T04:58:30.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341123+0000", 2024-09-13T04:58:30.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046690597 2024-09-13T04:58:30.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341165+0000", 2024-09-13T04:58:30.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2289000000000001e-05 2024-09-13T04:58:30.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341217+0000", 2024-09-13T04:58:30.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1484999999999999e-05 2024-09-13T04:58:30.852 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.852 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.853 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17901 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.853 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:30.853 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393735017999999, 2024-09-13T04:58:30.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046918842000000002, 2024-09-13T04:58:30.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17901 2024-09-13T04:58:30.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.855 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:30.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:30.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:30.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294438+0000", 2024-09-13T04:58:30.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9599999999999999e-06 2024-09-13T04:58:30.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294444+0000", 2024-09-13T04:58:30.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7740000000000001e-06 2024-09-13T04:58:30.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294448+0000", 2024-09-13T04:58:30.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5210000000000004e-06 2024-09-13T04:58:30.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341242+0000", 2024-09-13T04:58:30.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046794009999999997 2024-09-13T04:58:30.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341288+0000", 2024-09-13T04:58:30.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.596e-05 2024-09-13T04:58:30.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341355+0000", 2024-09-13T04:58:30.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6931e-05 2024-09-13T04:58:30.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.863 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.864 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.864 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17904 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.864 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:30.865 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393718977000001, 2024-09-13T04:58:30.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006963999999998, 2024-09-13T04:58:30.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.865 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.865 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17904 2024-09-13T04:58:30.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.866 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:30.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:30.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:30.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294454+0000", 2024-09-13T04:58:30.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.979e-06 2024-09-13T04:58:30.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294460+0000", 2024-09-13T04:58:30.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7899999999999996e-06 2024-09-13T04:58:30.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294464+0000", 2024-09-13T04:58:30.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4139999999999996e-06 2024-09-13T04:58:30.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341362+0000", 2024-09-13T04:58:30.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046897498000000003 2024-09-13T04:58:30.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341407+0000", 2024-09-13T04:58:30.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4839000000000002e-05 2024-09-13T04:58:30.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341459+0000", 2024-09-13T04:58:30.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2719000000000001e-05 2024-09-13T04:58:30.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.874 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17911 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.875 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:30.875 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393703079, 2024-09-13T04:58:30.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047111301000000001, 2024-09-13T04:58:30.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17911 2024-09-13T04:58:30.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:30.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:30.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:30.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294470+0000", 2024-09-13T04:58:30.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9769999999999999e-06 2024-09-13T04:58:30.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294476+0000", 2024-09-13T04:58:30.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.925e-06 2024-09-13T04:58:30.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294480+0000", 2024-09-13T04:58:30.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5390000000000001e-06 2024-09-13T04:58:30.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341487+0000", 2024-09-13T04:58:30.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006829 2024-09-13T04:58:30.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341530+0000", 2024-09-13T04:58:30.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2503000000000003e-05 2024-09-13T04:58:30.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341580+0000", 2024-09-13T04:58:30.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9848000000000001e-05 2024-09-13T04:58:30.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.884 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.885 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17914 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2048000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.885 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:30.885 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393687027, 2024-09-13T04:58:30.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047212684999999997, 2024-09-13T04:58:30.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17914 2024-09-13T04:58:30.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.887 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:30.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:30.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:30.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294486+0000", 2024-09-13T04:58:30.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0899999999999999e-06 2024-09-13T04:58:30.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294492+0000", 2024-09-13T04:58:30.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1920000000000002e-06 2024-09-13T04:58:30.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294497+0000", 2024-09-13T04:58:30.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5399999999999997e-06 2024-09-13T04:58:30.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341606+0000", 2024-09-13T04:58:30.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047108783000000001 2024-09-13T04:58:30.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341648+0000", 2024-09-13T04:58:30.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1878999999999997e-05 2024-09-13T04:58:30.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341697+0000", 2024-09-13T04:58:30.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9521999999999998e-05 2024-09-13T04:58:30.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.897 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.898 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17916 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2088960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.898 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:30.898 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.393670220000001, 2024-09-13T04:58:30.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047303235999999999, 2024-09-13T04:58:30.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:30.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17916 2024-09-13T04:58:30.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.900 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:30.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:30.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:30.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:30.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294506+0000", 2024-09-13T04:58:30.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1880000000000001e-06 2024-09-13T04:58:30.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294518+0000", 2024-09-13T04:58:30.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1802e-05 2024-09-13T04:58:30.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294523+0000", 2024-09-13T04:58:30.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7219999999999999e-06 2024-09-13T04:58:30.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341713+0000", 2024-09-13T04:58:30.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190086999999999 2024-09-13T04:58:30.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341755+0000", 2024-09-13T04:58:30.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2530000000000001e-05 2024-09-13T04:58:30.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341804+0000", 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9245e-05 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.232520078, 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:58:30.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:58:30.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:58:30.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.67492043099999999, 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:58:30.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:58:30.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:58:30.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:30.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.66040937899999996, 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:30.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:58:30.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:30.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:30.927 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:30.928 INFO:teuthology.orchestra.run.smithi001.stdout:1476 read 0x9d8c74 thru 0x9e0e94 (0x8221 bytes) 2024-09-13T04:58:30.928 INFO:teuthology.orchestra.run.smithi001.stdout:1477 read 0x5de9a47 thru 0x5dee970 (0x4f2a bytes) 2024-09-13T04:58:30.928 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.928 INFO:teuthology.orchestra.run.smithi001.stdout:1479 write 0x3d3fca2 thru 0x3d47259 (0x75b8 bytes) 2024-09-13T04:58:30.929 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:30.930 INFO:teuthology.orchestra.run.smithi001.stdout:1480 write 0x8bed6dd thru 0x8bf2e50 (0x5774 bytes) 2024-09-13T04:58:30.930 INFO:teuthology.orchestra.run.smithi001.stdout:1481 punch from 0x33c8632 to 0x33d2a12, (0xa3e0 bytes) 2024-09-13T04:58:30.930 INFO:teuthology.orchestra.run.smithi001.stdout:1483 write 0x23866f3 thru 0x238740a (0xd18 bytes) 2024-09-13T04:58:30.930 INFO:teuthology.orchestra.run.smithi001.stdout:1484 write 0xd7f101f thru 0xd7f5974 (0x4956 bytes) 2024-09-13T04:58:30.931 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:58:30.931 INFO:teuthology.orchestra.run.smithi001.stdout:1485 trunc from 0xd7f5975 to 0x9710ac4 2024-09-13T04:58:30.932 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:30.772+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1486 write 0xdb96ef3 thru 0xdba1eb9 (0xafc7 bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1487 read 0x90764db thru 0x90848ec (0xe412 bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1489 read 0xa6014b1 thru 0xa60a6a6 (0x91f6 bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1490 read 0xda64b5f thru 0xda6f83a (0xacdc bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1491 write 0x10fb3b7 thru 0x10fb884 (0x4ce bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1492 write 0x4910e21 thru 0x4913ec6 (0x30a6 bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1493 write 0xb838965 thru 0xb83e586 (0x5c22 bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1494 write 0xb617b6b thru 0xb625639 (0xdacf bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1496 writesame 0xa55d22a thru 0xa55deee data_size 0x1(0xcc5 bytes) 2024-09-13T04:58:30.935 INFO:teuthology.orchestra.run.smithi001.stdout:1497 read 0x82bffc0 thru 0x82c68ea (0x692b bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1498 trunc from 0xdba1eba to 0xabdb24f 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1500 write 0x49c590 thru 0x49e5bf (0x2030 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1501 writesame 0xbd77c09 thru 0xbd7871e data_size 0xb(0xb16 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1503 write 0xb6855 thru 0xc4d3f (0xe4eb bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1507 read 0x25c7251 thru 0x25ce316 (0x70c6 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1509 write 0x3a99ae0 thru 0x3a9e770 (0x4c91 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1510 punch from 0x2d7ae7f to 0x2d7c2e6, (0x1467 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1511 write 0xa6f56bb thru 0xa6fedde (0x9724 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1514 write 0x6d997ba thru 0x6d9f71d (0x5f64 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1516 punch from 0xb020ac9 to 0xb02a11b, (0x9652 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:1518 write 0x6786f99 thru 0x678eb1c (0x7b84 bytes) 2024-09-13T04:58:30.936 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.937 INFO:teuthology.orchestra.run.smithi001.stdout:1521 writesame 0x5d5883b thru 0x5d66baf data_size 0x1(0xe375 bytes) 2024-09-13T04:58:30.979 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:30.976+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:30.988 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:30.989 INFO:teuthology.orchestra.run.smithi001.stdout:1524 punch from 0x3f30bee to 0x3f3d26d, (0xc67f bytes) 2024-09-13T04:58:30.989 INFO:teuthology.orchestra.run.smithi001.stdout:1526 punch from 0xa53738 to 0xa557ec, (0x20b4 bytes) 2024-09-13T04:58:30.991 INFO:teuthology.orchestra.run.smithi001.stdout:1528 trunc from 0xbd7871f to 0xae458dc 2024-09-13T04:58:31.007 INFO:teuthology.orchestra.run.smithi001.stdout:1530 read 0x3842c25 thru 0x385129f (0xe67b bytes) 2024-09-13T04:58:31.022 INFO:teuthology.orchestra.run.smithi001.stdout:1532 writesame 0xcd7b37e thru 0xcd8133b data_size 0x72(0x5fbe bytes) 2024-09-13T04:58:31.033 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:31.031+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:31.034 INFO:teuthology.orchestra.run.smithi001.stdout:1533 writesame 0xd49f1a3 thru 0xd4ac38e data_size 0xa(0xd1ec bytes) 2024-09-13T04:58:31.052 INFO:teuthology.orchestra.run.smithi001.stdout:1534 trunc from 0xd4ac38f to 0x40c4813 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:41165 3.19 3:9e9a16eb:::rbd_header.112aa73d3d8d:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:31.061101+0000", 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00020726300000000001, 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025159100000000001, 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 41165 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:31.061101+0000", 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:31.061101+0000", 2024-09-13T04:58:31.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:31.061099+0000", 2024-09-13T04:58:31.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:31.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:31.061112+0000", 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2792999999999999e-05 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:31.061141+0000", 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9581999999999999e-05 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:31.061155+0000", 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3339000000000001e-05 2024-09-13T04:58:31.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:31.061267+0000", 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000112543 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-13T04:58:31.065 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:31.070 INFO:teuthology.orchestra.run.smithi001.stdout:1535 read 0x2f2062e thru 0x2f23dcb (0x379e bytes) 2024-09-13T04:58:31.070 INFO:teuthology.orchestra.run.smithi001.stdout:1536 read 0x30b4496 thru 0x30c07b6 (0xc321 bytes) 2024-09-13T04:58:31.071 INFO:teuthology.orchestra.run.smithi001.stdout:1537 trunc from 0x40c4813 to 0x4200d41 2024-09-13T04:58:31.076 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:31.090 INFO:teuthology.orchestra.run.smithi001.stdout:1539 trunc from 0x4200d41 to 0x6514d5 2024-09-13T04:58:31.134 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:31.132+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:31.137 INFO:teuthology.orchestra.run.smithi001.stdout:1540 write 0x9496ac6 thru 0x9498b1d (0x2058 bytes) 2024-09-13T04:58:31.141 INFO:teuthology.orchestra.run.smithi001.stdout:1543 write 0x21da11f thru 0x21dd774 (0x3656 bytes) 2024-09-13T04:58:31.144 INFO:teuthology.orchestra.run.smithi001.stdout:1544 write 0x6c0c6a3 thru 0x6c1bbd7 (0xf535 bytes) 2024-09-13T04:58:31.157 INFO:teuthology.orchestra.run.smithi001.stdout:1545 trunc from 0x9498b1e to 0x604052c 2024-09-13T04:58:31.164 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.174 INFO:teuthology.orchestra.run.smithi001.stdout:1548 write 0x8ef37ca thru 0x8efd444 (0x9c7b bytes) 2024-09-13T04:58:31.177 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.177 INFO:teuthology.orchestra.run.smithi001.stdout:1550 writesame 0x58b7302 thru 0x58bbf6f data_size 0x3(0x4c6e bytes) 2024-09-13T04:58:31.182 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.182 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.182 INFO:teuthology.orchestra.run.smithi001.stdout:1554 read 0x5e875b thru 0x5f78a2 (0xf148 bytes) 2024-09-13T04:58:31.183 INFO:teuthology.orchestra.run.smithi001.stdout:1555 punch from 0x88ebfee to 0x88eedac, (0x2dbe bytes) 2024-09-13T04:58:31.190 INFO:teuthology.orchestra.run.smithi001.stdout:1556 write 0x99e031f thru 0x99e9483 (0x9165 bytes) 2024-09-13T04:58:31.193 INFO:teuthology.orchestra.run.smithi001.stdout:1558 writesame 0x3e133f0 thru 0x3e233ef data_size 0x20(0x10000 bytes) 2024-09-13T04:58:31.197 INFO:teuthology.orchestra.run.smithi001.stdout:1560 writesame 0x48bf9e4 thru 0x48cf78e data_size 0x1(0xfdab bytes) 2024-09-13T04:58:31.230 INFO:teuthology.orchestra.run.smithi001.stdout:1561 read 0x7ae6b6d thru 0x7af25d7 (0xba6b bytes) 2024-09-13T04:58:31.230 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.230 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.235 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:31.232+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:31.246 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:31.246 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:31.253 INFO:teuthology.orchestra.run.smithi001.stdout:1565 writesame 0xd37f319 thru 0xd38a948 data_size 0x58(0xb630 bytes) 2024-09-13T04:58:31.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:31.262 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:31.264 INFO:teuthology.orchestra.run.smithi001.stdout:1567 write 0xd746ba4 thru 0xd7486cc (0x1b29 bytes) 2024-09-13T04:58:31.269 INFO:teuthology.orchestra.run.smithi001.stdout:1568 write 0xd2ed605 thru 0xd2f1b4a (0x4546 bytes) 2024-09-13T04:58:31.271 INFO:teuthology.orchestra.run.smithi001.stdout:1570 read 0x8dfd9fd thru 0x8e01059 (0x365d bytes) 2024-09-13T04:58:31.297 INFO:teuthology.orchestra.run.smithi001.stdout:1572 trunc from 0xd7486cd to 0x274586d 2024-09-13T04:58:31.324 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.324 INFO:teuthology.orchestra.run.smithi001.stdout:1575 read 0x6bfd09 thru 0x6c9093 (0x938b bytes) 2024-09-13T04:58:31.325 INFO:teuthology.orchestra.run.smithi001.stdout:1576 read 0x1fc61b3 thru 0x1fcfd07 (0x9b55 bytes) 2024-09-13T04:58:31.336 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:31.332+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:31.345 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:31.346 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:58:31.354 INFO:teuthology.orchestra.run.smithi001.stdout:1578 write 0xe210b27 thru 0xe211b98 (0x1072 bytes) 2024-09-13T04:58:31.357 INFO:teuthology.orchestra.run.smithi001.stdout:1579 write 0xda9e488 thru 0xdaa78b0 (0x9429 bytes) 2024-09-13T04:58:31.366 INFO:teuthology.orchestra.run.smithi001.stdout:1581 read 0xb0803c2 thru 0xb086275 (0x5eb4 bytes) 2024-09-13T04:58:31.366 INFO:teuthology.orchestra.run.smithi001.stdout:1582 writesame 0x2bdb3f thru 0x2c9ae4 data_size 0x4e(0xbfa6 bytes) 2024-09-13T04:58:31.368 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-09-13T04:58:30.917111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":213418676,"num_objects":489,"num_object_clones":153,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":237,"num_read":47016,"num_read_kb":128772,"num_write":3861,"num_write_kb":83851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3183,"ondisk_log_size":3183,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279600,"kb_used_data":14320,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754695120,"statfs":{"total":773094113280,"available":772807802880,"internally_reserved":0,"allocated":14663680,"data_stored":101757836,"data_compressed":2608819,"data_compressed_allocated":7757824,"data_compressed_original":97750138,"omap_allocated":12743,"internal_metadata":271568441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[18,49,4,0,81,103],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16012627,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8285,"num_read_kb":42844,"num_write":678,"num_write_kb":6948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000909"},"pg_stats":[{"pgid":"3.1f","version":"41'107","reported_seq":1625,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.763089+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:58:26.763089+0000","last_peered":"2024-09-13T04:58:26.763089+0000","last_clean":"2024-09-13T04:58:26.763089+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:58:26.763089+0000","last_undegraded":"2024-09-13T04:58:26.763089+0000","last_fullsized":"2024-09-13T04:58:26.763089+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":21111752,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":1880,"num_read_kb":11182,"num_write":121,"num_write_kb":6008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"41'49","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.790763+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:58:26.790763+0000","last_peered":"2024-09-13T04:58:26.790763+0000","last_clean":"2024-09-13T04:58:26.790763+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:58:26.790763+0000","last_undegraded":"2024-09-13T04:58:26.790763+0000","last_fullsized":"2024-09-13T04:58:26.790763+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":1507173,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":7,"num_read":28,"num_read_kb":89,"num_write":40,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"41'70","reported_seq":138,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.816529+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:58:26.816529+0000","last_peered":"2024-09-13T04:58:26.816529+0000","last_clean":"2024-09-13T04:58:26.816529+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:58:26.816529+0000","last_undegraded":"2024-09-13T04:58:26.816529+0000","last_fullsized":"2024-09-13T04:58:26.816529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":15073970,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":9,"num_read":58,"num_read_kb":264,"num_write":60,"num_write_kb":688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"42'50","reported_seq":167,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.777451+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:58:27.777451+0000","last_peered":"2024-09-13T04:58:27.777451+0000","last_clean":"2024-09-13T04:58:27.777451+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:58:27.777451+0000","last_undegraded":"2024-09-13T04:58:27.777451+0000","last_fullsized":"2024-09-13T04:58:27.777451+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":1720320,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":120,"num_read_kb":690,"num_write":45,"num_write_kb":775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"41'112","reported_seq":4083,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.276307+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:58:26.276307+0000","last_peered":"2024-09-13T04:58:26.276307+0000","last_clean":"2024-09-13T04:58:26.276307+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:58:26.276307+0000","last_undegraded":"2024-09-13T04:58:26.276307+0000","last_fullsized":"2024-09-13T04:58:26.276307+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":5824512,"num_objects":16,"num_object_clones":7,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":4483,"num_read_kb":8602,"num_write":97,"num_write_kb":1114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"39'57","reported_seq":730,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.866805+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:58:27.866805+0000","last_peered":"2024-09-13T04:58:27.866805+0000","last_clean":"2024-09-13T04:58:27.866805+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:58:27.866805+0000","last_undegraded":"2024-09-13T04:58:27.866805+0000","last_fullsized":"2024-09-13T04:58:27.866805+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":9107392,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":671,"num_read_kb":2106,"num_write":48,"num_write_kb":808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"42'102","reported_seq":2734,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.853208+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:58:27.853208+0000","last_peered":"2024-09-13T04:58:27.853208+0000","last_clean":"2024-09-13T04:58:27.853208+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:58:27.853208+0000","last_undegraded":"2024-09-13T04:58:27.853208+0000","last_fullsized":"2024-09-13T04:58:27.853208+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":9010379,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":2674,"num_read_kb":6472,"num_write":88,"num_write_kb":2783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"41'78","reported_seq":1681,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.769389+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:58:26.769389+0000","last_peered":"2024-09-13T04:58:26.769389+0000","last_clean":"2024-09-13T04:58:26.769389+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:58:26.769389+0000","last_undegraded":"2024-09-13T04:58:26.769389+0000","last_fullsized":"2024-09-13T04:58:26.769389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":15353347,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":1637,"num_read_kb":2284,"num_write":73,"num_write_kb":633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"41'65","reported_seq":2495,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.522462+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:58:26.522462+0000","last_peered":"2024-09-13T04:58:26.522462+0000","last_clean":"2024-09-13T04:58:26.522462+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:58:26.522462+0000","last_undegraded":"2024-09-13T04:58:26.522462+0000","last_fullsized":"2024-09-13T04:58:26.522462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":14098580,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":2428,"num_read_kb":15061,"num_write":83,"num_write_kb":11021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"41'265","reported_seq":311,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.817134+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:58:26.817134+0000","last_peered":"2024-09-13T04:58:26.817134+0000","last_clean":"2024-09-13T04:58:26.817134+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:58:26.817134+0000","last_undegraded":"2024-09-13T04:58:26.817134+0000","last_fullsized":"2024-09-13T04:58:26.817134+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":3645547,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":1064,"num_read_kb":833,"num_write":392,"num_write_kb":2134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"41'29","reported_seq":78,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:25.222556+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:58:25.222556+0000","last_peered":"2024-09-13T04:58:25.222556+0000","last_clean":"2024-09-13T04:58:25.222556+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:58:25.222556+0000","last_undegraded":"2024-09-13T04:58:25.222556+0000","last_fullsized":"2024-09-13T04:58:25.222556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":1104361,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":31,"num_read_kb":16,"num_write":25,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"41'66","reported_seq":2276,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.625220+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:58:24.625220+0000","last_peered":"2024-09-13T04:58:24.625220+0000","last_clean":"2024-09-13T04:58:24.625220+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:58:24.625220+0000","last_undegraded":"2024-09-13T04:58:24.625220+0000","last_fullsized":"2024-09-13T04:58:24.625220+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":5093834,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":2408,"num_read_kb":5404,"num_write":52,"num_write_kb":247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.340007+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:58:24.340007+0000","last_peered":"2024-09-13T04:58:24.340007+0000","last_clean":"2024-09-13T04:58:24.340007+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:58:24.340007+0000","last_undegraded":"2024-09-13T04:58:24.340007+0000","last_fullsized":"2024-09-13T04:58:24.340007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"41'44","reported_seq":1297,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.785272+0000","last_change":"2024-09-13T04:58:19.436873+0000","last_active":"2024-09-13T04:58:26.785272+0000","last_peered":"2024-09-13T04:58:26.785272+0000","last_clean":"2024-09-13T04:58:26.785272+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T04:58:26.785272+0000","last_undegraded":"2024-09-13T04:58:26.785272+0000","last_fullsized":"2024-09-13T04:58:26.785272+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":9678903,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":1274,"num_read_kb":10063,"num_write":64,"num_write_kb":11437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.327435+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:58:24.327435+0000","last_peered":"2024-09-13T04:58:24.327435+0000","last_clean":"2024-09-13T04:58:24.327435+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:58:24.327435+0000","last_undegraded":"2024-09-13T04:58:24.327435+0000","last_fullsized":"2024-09-13T04:58:24.327435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"42'119","reported_seq":585,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.867568+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:58:27.867568+0000","last_peered":"2024-09-13T04:58:27.867568+0000","last_clean":"2024-09-13T04:58:27.867568+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:58:27.867568+0000","last_undegraded":"2024-09-13T04:58:27.867568+0000","last_fullsized":"2024-09-13T04:58:27.867568+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":942127,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":1249,"num_read_kb":1616,"num_write":111,"num_write_kb":1311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.340336+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:58:24.340336+0000","last_peered":"2024-09-13T04:58:24.340336+0000","last_clean":"2024-09-13T04:58:24.340336+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:58:24.340336+0000","last_undegraded":"2024-09-13T04:58:24.340336+0000","last_fullsized":"2024-09-13T04:58:24.340336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"39'33","reported_seq":77,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.341707+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:58:24.341707+0000","last_peered":"2024-09-13T04:58:24.341707+0000","last_clean":"2024-09-13T04:58:24.341707+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:58:24.341707+0000","last_undegraded":"2024-09-13T04:58:24.341707+0000","last_fullsized":"2024-09-13T04:58:24.341707+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":1736704,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":30,"num_read_kb":20,"num_write":29,"num_write_kb":786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T04:58:23.337329+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:58:23.337329+0000","last_peered":"2024-09-13T04:58:23.337329+0000","last_clean":"2024-09-13T04:58:23.337329+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:58:23.337329+0000","last_undegraded":"2024-09-13T04:58:23.337329+0000","last_fullsized":"2024-09-13T04:58:23.337329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"40'170","reported_seq":229,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.876289+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:58:27.876289+0000","last_peered":"2024-09-13T04:58:27.876289+0000","last_clean":"2024-09-13T04:58:27.876289+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:58:27.876289+0000","last_undegraded":"2024-09-13T04:58:27.876289+0000","last_fullsized":"2024-09-13T04:58:27.876289+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":1245341,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":732,"num_read_kb":571,"num_write":260,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"42'122","reported_seq":1898,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.886511+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:58:27.886511+0000","last_peered":"2024-09-13T04:58:27.886511+0000","last_clean":"2024-09-13T04:58:27.886511+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:58:27.886511+0000","last_undegraded":"2024-09-13T04:58:27.886511+0000","last_fullsized":"2024-09-13T04:58:27.886511+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":15699495,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":10,"num_read":2351,"num_read_kb":11348,"num_write":117,"num_write_kb":9865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.340424+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:58:24.340424+0000","last_peered":"2024-09-13T04:58:24.340424+0000","last_clean":"2024-09-13T04:58:24.340424+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:58:24.340424+0000","last_undegraded":"2024-09-13T04:58:24.340424+0000","last_fullsized":"2024-09-13T04:58:24.340424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"41'269","reported_seq":320,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.814231+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:58:26.814231+0000","last_peered":"2024-09-13T04:58:26.814231+0000","last_clean":"2024-09-13T04:58:26.814231+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:58:26.814231+0000","last_undegraded":"2024-09-13T04:58:26.814231+0000","last_fullsized":"2024-09-13T04:58:26.814231+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":521335,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":1155,"num_read_kb":923,"num_write":420,"num_write_kb":1020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.336262+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:58:24.336262+0000","last_peered":"2024-09-13T04:58:24.336262+0000","last_clean":"2024-09-13T04:58:24.336262+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:58:24.336262+0000","last_undegraded":"2024-09-13T04:58:24.336262+0000","last_fullsized":"2024-09-13T04:58:24.336262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"41'49","reported_seq":111,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.794338+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:58:26.794338+0000","last_peered":"2024-09-13T04:58:26.794338+0000","last_clean":"2024-09-13T04:58:26.794338+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:58:26.794338+0000","last_undegraded":"2024-09-13T04:58:26.794338+0000","last_fullsized":"2024-09-13T04:58:26.794338+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":8176689,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":53,"num_read_kb":786,"num_write":46,"num_write_kb":1694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.339598+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:58:24.339598+0000","last_peered":"2024-09-13T04:58:24.339598+0000","last_clean":"2024-09-13T04:58:24.339598+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:58:24.339598+0000","last_undegraded":"2024-09-13T04:58:24.339598+0000","last_fullsized":"2024-09-13T04:58:24.339598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"39'19","reported_seq":64,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.327169+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:58:24.327169+0000","last_peered":"2024-09-13T04:58:24.327169+0000","last_clean":"2024-09-13T04:58:24.327169+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:58:24.327169+0000","last_undegraded":"2024-09-13T04:58:24.327169+0000","last_fullsized":"2024-09-13T04:58:24.327169+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":1048592,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":26,"num_read_kb":649,"num_write":22,"num_write_kb":895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.340047+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:58:24.340047+0000","last_peered":"2024-09-13T04:58:24.340047+0000","last_clean":"2024-09-13T04:58:24.340047+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:58:24.340047+0000","last_undegraded":"2024-09-13T04:58:24.340047+0000","last_fullsized":"2024-09-13T04:58:24.340047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.343724+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:58:24.343724+0000","last_peered":"2024-09-13T04:58:24.343724+0000","last_clean":"2024-09-13T04:58:24.343724+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:58:24.343724+0000","last_undegraded":"2024-09-13T04:58:24.343724+0000","last_fullsized":"2024-09-13T04:58:24.343724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"41'50","reported_seq":325,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:25.839673+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:58:25.839673+0000","last_peered":"2024-09-13T04:58:25.839673+0000","last_clean":"2024-09-13T04:58:25.839673+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:58:25.839673+0000","last_undegraded":"2024-09-13T04:58:25.839673+0000","last_fullsized":"2024-09-13T04:58:25.839673+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":2027536,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":749,"num_read_kb":1239,"num_write":38,"num_write_kb":717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"41'178","reported_seq":2107,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:25.978469+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:58:25.978469+0000","last_peered":"2024-09-13T04:58:25.978469+0000","last_clean":"2024-09-13T04:58:25.978469+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:58:25.978469+0000","last_undegraded":"2024-09-13T04:58:25.978469+0000","last_fullsized":"2024-09-13T04:58:25.978469+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":6021918,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":11,"num_read":2548,"num_read_kb":5294,"num_write":254,"num_write_kb":1078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"42'103","reported_seq":1726,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.728898+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:58:27.728898+0000","last_peered":"2024-09-13T04:58:27.728898+0000","last_clean":"2024-09-13T04:58:27.728898+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:58:27.728898+0000","last_undegraded":"2024-09-13T04:58:27.728898+0000","last_fullsized":"2024-09-13T04:58:27.728898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":5914659,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":1773,"num_read_kb":3337,"num_write":117,"num_write_kb":1177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"41'30","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.812608+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:58:26.812608+0000","last_peered":"2024-09-13T04:58:26.812608+0000","last_clean":"2024-09-13T04:58:26.812608+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:58:26.812608+0000","last_undegraded":"2024-09-13T04:58:26.812608+0000","last_fullsized":"2024-09-13T04:58:26.812608+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":954384,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":26,"num_read_kb":55,"num_write":25,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"41'298","reported_seq":349,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:25.636170+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:58:25.636170+0000","last_peered":"2024-09-13T04:58:25.636170+0000","last_clean":"2024-09-13T04:58:25.636170+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:58:25.636170+0000","last_undegraded":"2024-09-13T04:58:25.636170+0000","last_fullsized":"2024-09-13T04:58:25.636170+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":1422248,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":9,"num_read":1449,"num_read_kb":1216,"num_write":510,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"39'136","reported_seq":826,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.340201+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:58:24.340201+0000","last_peered":"2024-09-13T04:58:24.340201+0000","last_clean":"2024-09-13T04:58:24.340201+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:58:24.340201+0000","last_undegraded":"2024-09-13T04:58:24.340201+0000","last_fullsized":"2024-09-13T04:58:24.340201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":3489322,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":1887,"num_read_kb":1653,"num_write":132,"num_write_kb":562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"41'49","reported_seq":91,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.824090+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:58:26.824090+0000","last_peered":"2024-09-13T04:58:26.824090+0000","last_clean":"2024-09-13T04:58:26.824090+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:58:26.824090+0000","last_undegraded":"2024-09-13T04:58:26.824090+0000","last_fullsized":"2024-09-13T04:58:26.824090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":8011428,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":22,"num_read_kb":105,"num_write":44,"num_write_kb":649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"42'51","reported_seq":4792,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.843239+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:58:27.843239+0000","last_peered":"2024-09-13T04:58:27.843239+0000","last_clean":"2024-09-13T04:58:27.843239+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:58:27.843239+0000","last_undegraded":"2024-09-13T04:58:27.843239+0000","last_fullsized":"2024-09-13T04:58:27.843239+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":11612176,"num_objects":19,"num_object_clones":8,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":4749,"num_read_kb":14040,"num_write":69,"num_write_kb":8422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"41'79","reported_seq":3734,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.795490+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:58:26.795490+0000","last_peered":"2024-09-13T04:58:26.795490+0000","last_clean":"2024-09-13T04:58:26.795490+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:58:26.795490+0000","last_undegraded":"2024-09-13T04:58:26.795490+0000","last_fullsized":"2024-09-13T04:58:26.795490+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":4786858,"num_objects":21,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":3761,"num_read_kb":5107,"num_write":86,"num_write_kb":254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"41'82","reported_seq":3010,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:26.816740+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:58:26.816740+0000","last_peered":"2024-09-13T04:58:26.816740+0000","last_clean":"2024-09-13T04:58:26.816740+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:58:26.816740+0000","last_undegraded":"2024-09-13T04:58:26.816740+0000","last_fullsized":"2024-09-13T04:58:26.816740+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":19485119,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":2944,"num_read_kb":14549,"num_write":103,"num_write_kb":12729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"42'49","reported_seq":1636,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T04:58:27.736082+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:58:27.736082+0000","last_peered":"2024-09-13T04:58:27.736082+0000","last_clean":"2024-09-13T04:58:27.736082+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:58:27.736082+0000","last_undegraded":"2024-09-13T04:58:27.736082+0000","last_fullsized":"2024-09-13T04:58:27.736082+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":1457196,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":1576,"num_read_kb":2114,"num_write":55,"num_write_kb":668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"38'83","reported_seq":494,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T04:58:24.339916+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:58:24.339916+0000","last_peered":"2024-09-13T04:58:24.339916+0000","last_clean":"2024-09-13T04:58:24.339916+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:58:24.339916+0000","last_undegraded":"2024-09-13T04:58:24.339916+0000","last_fullsized":"2024-09-13T04:58:24.339916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":5945092,"num_objects":21,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":1064,"num_read_kb":984,"num_write":74,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":212828289,"num_objects":485,"num_object_clones":153,"num_object_copies":970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":485,"num_whiteouts":237,"num_read":46900,"num_read_kb":128672,"num_write":3700,"num_write_kb":81433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10878976,"data_stored":99015566,"data_compressed":2558412,"data_compressed_allocated":7643136,"data_compressed_original":96545914,"omap_allocated":0,"internal_metadata":0},"log_size":3063,"ondisk_log_size":3063,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":593920,"data_stored":904240,"data_compressed":18072,"data_compressed_allocated":49152,"data_compressed_original":659456,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1495040,"data_stored":8229801,"data_compressed":184296,"data_compressed_allocated":675840,"data_compressed_original":7778304,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1585152,"data_stored":11227412,"data_compressed":231231,"data_compressed_allocated":831488,"data_compressed_original":10808747,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37576,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334264,"statfs":{"total":96636764160,"available":96598286336,"internally_reserved":0,"allocated":2424832,"data_stored":18565872,"data_compressed":448030,"data_compressed_allocated":1359872,"data_compressed_original":17907656,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]}]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2199552,"data_stored":19564285,"data_compressed":456059,"data_compressed_allocated":1396736,"data_compressed_original":19131470,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37388,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2232320,"data_stored":15263507,"data_compressed":438342,"data_compressed_allocated":1228800,"data_compressed_original":14644225,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37044,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334796,"statfs":{"total":96636764160,"available":96598831104,"internally_reserved":0,"allocated":1880064,"data_stored":13959883,"data_compressed":367105,"data_compressed_allocated":1015808,"data_compressed_original":13433034,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[18,49,4,0,81,103],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.474}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.834}]}]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2252800,"data_stored":14042836,"data_compressed":465684,"data_compressed_allocated":1200128,"data_compressed_original":13387246,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1433600,"data_stored":13772685,"data_compressed":367105,"data_compressed_allocated":1015808,"data_compressed_original":13433034,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1724416,"data_stored":14485904,"data_compressed":432115,"data_compressed_allocated":1191936,"data_compressed_original":14070785,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1748992,"data_stored":19377071,"data_compressed":456059,"data_compressed_allocated":1396736,"data_compressed_original":19131470,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1970176,"data_stored":18378657,"data_compressed":448030,"data_compressed_allocated":1359872,"data_compressed_original":17907656,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1130496,"data_stored":11040161,"data_compressed":231231,"data_compressed_allocated":831488,"data_compressed_original":10808747,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1691648,"data_stored":13201513,"data_compressed":421504,"data_compressed_allocated":1122304,"data_compressed_original":12756462,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1040384,"data_stored":8042586,"data_compressed":184296,"data_compressed_allocated":675840,"data_compressed_original":7778304,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":716989,"data_compressed":18072,"data_compressed_allocated":49152,"data_compressed_original":659456,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:58:31.370 INFO:tasks.thrashosds.thrasher:Setting 3.1e to [3, 4, 7, 0] 2024-09-13T04:58:31.370 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1e', '3', '4', '7', '0'] 2024-09-13T04:58:31.370 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1e 3 4 7 0 2024-09-13T04:58:31.372 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.372 INFO:teuthology.orchestra.run.smithi001.stdout:1587 read 0xe039fc9 thru 0xe046602 (0xc63a bytes) 2024-09-13T04:58:31.372 INFO:teuthology.orchestra.run.smithi001.stdout:1588 punch from 0x517e46a to 0x518c6b7, (0xe24d bytes) 2024-09-13T04:58:31.372 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.372 INFO:teuthology.orchestra.run.smithi001.stdout:1592 read 0xf632b5 thru 0xf712dd (0xe029 bytes) 2024-09-13T04:58:31.372 INFO:teuthology.orchestra.run.smithi001.stdout:1594 writesame 0x5c9094b thru 0x5c9aa4f data_size 0x1(0xa105 bytes) 2024-09-13T04:58:31.388 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:31.388 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:31.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15997 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1314816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360845355, 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088423697999999995, 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.389 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15997 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:31.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026558+0000", 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026564+0000", 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4399999999999996e-06 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026580+0000", 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6149000000000001e-05 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026593+0000", 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2723e-05 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114730+0000", 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088137005000000004 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114857+0000", 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012730799999999999 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114983+0000", 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012595600000000001 2024-09-13T04:58:31.393 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15998 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360800827, 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088737302000000004, 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15998 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026603+0000", 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026608+0000", 2024-09-13T04:58:31.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7440000000000004e-06 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026626+0000", 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8114999999999999e-05 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026638+0000", 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1976e-05 2024-09-13T04:58:31.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115046+0000", 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088407311000000002 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115180+0000", 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013447700000000001 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115341+0000", 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000160676 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15999 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360754632999999, 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089223035000000006, 2024-09-13T04:58:31.399 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15999 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026649+0000", 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026655+0000", 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9560000000000002e-06 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026671+0000", 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026684+0000", 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.273e-05 2024-09-13T04:58:31.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115423+0000", 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088738368999999997 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115651+0000", 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022799699999999999 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115873+0000", 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222337 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16000 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360708675, 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089511479000000005, 2024-09-13T04:58:31.404 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16000 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:31.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026695+0000", 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026701+0000", 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9000000000000003e-06 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026717+0000", 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6379e-05 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026731+0000", 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4039000000000001e-05 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115965+0000", 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089233554000000007 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116078+0000", 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000113655 2024-09-13T04:58:31.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116207+0000", 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128942 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16001 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1478656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360615709999999, 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089757132000000003, 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16001 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026788+0000", 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026794+0000", 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8059999999999997e-06 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026827+0000", 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3002e-05 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026842+0000", 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4516e-05 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116278+0000", 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089435635999999999 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116414+0000", 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000135919 2024-09-13T04:58:31.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116546+0000", 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013235300000000001 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16002 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1519616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:31.414 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360548743000001, 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089990789000000002, 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16002 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026854+0000", 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:31.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026862+0000", 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5789999999999998e-06 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026883+0000", 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1892e-05 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026897+0000", 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.326e-05 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116609+0000", 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089712560999999996 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116708+0000", 2024-09-13T04:58:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8787000000000001e-05 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116846+0000", 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013842999999999999 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16003 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.419 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360496116, 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090223747000000007, 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16003 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026907+0000", 2024-09-13T04:58:31.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026913+0000", 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026930+0000", 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7065999999999999e-05 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026943+0000", 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3249e-05 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116903+0000", 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089959340999999998 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116992+0000", 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9690000000000004e-05 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117132+0000", 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013984800000000001 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16004 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1601536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360436913000001, 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090483628999999996, 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16004 2024-09-13T04:58:31.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026967+0000", 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026978+0000", 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1219e-05 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027011+0000", 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3222000000000001e-05 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027025+0000", 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3995999999999999e-05 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117203+0000", 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090178122999999999 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117287+0000", 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3721999999999996e-05 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117451+0000", 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016434300000000001 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16005 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360366022999999, 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090781484999999995, 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.430 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16005 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027037+0000", 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027044+0000", 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4640000000000003e-06 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027061+0000", 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6949e-05 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027074+0000", 2024-09-13T04:58:31.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3422e-05 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117547+0000", 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.09047318 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117682+0000", 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000134602 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117820+0000", 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013803200000000001 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16006 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360316921999999, 2024-09-13T04:58:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091107120999999999, 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16006 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027086+0000", 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027092+0000", 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027109+0000", 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6330000000000001e-05 2024-09-13T04:58:31.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027119+0000", 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0132999999999999e-05 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117876+0000", 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090756823 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118024+0000", 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000148758 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118195+0000", 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017021500000000001 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.440 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16007 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1724416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360278638, 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091267529999999999, 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.441 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16007 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027125+0000", 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027129+0000", 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5599999999999998e-06 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027137+0000", 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7129999999999994e-06 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027144+0000", 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8329999999999996e-06 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118240+0000", 2024-09-13T04:58:31.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091096166000000006 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118313+0000", 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000005e-05 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118393+0000", 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0680999999999995e-05 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.445 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16008 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1765376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360254133, 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091580432000000003, 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16008 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027153+0000", 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3500000000000001e-06 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027161+0000", 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1799999999999996e-06 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027168+0000", 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5919999999999997e-06 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118454+0000", 2024-09-13T04:58:31.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091286415999999995 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118592+0000", 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013743699999999999 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118731+0000", 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013906300000000001 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.450 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16009 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360226239999999, 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091763040000000004, 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16009 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:31.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027177+0000", 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027181+0000", 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7160000000000001e-06 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027190+0000", 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4630000000000001e-06 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027196+0000", 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6429999999999998e-06 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118776+0000", 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091579259999999996 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118855+0000", 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9695000000000002e-05 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118941+0000", 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6013000000000005e-05 2024-09-13T04:58:31.455 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16010 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1847296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360202375, 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091942769999999993, 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16010 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027201+0000", 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027205+0000", 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.213e-06 2024-09-13T04:58:31.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027213+0000", 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2460000000000003e-06 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027220+0000", 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9970000000000001e-06 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118986+0000", 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091766174000000006 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119056+0000", 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9988999999999997e-05 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119145+0000", 2024-09-13T04:58:31.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8751000000000005e-05 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16011 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1888256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360178400000001, 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130156000000005, 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16011 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027225+0000", 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027229+0000", 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1640000000000001e-06 2024-09-13T04:58:31.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027237+0000", 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3299999999999999e-06 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027243+0000", 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4620000000000001e-06 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119188+0000", 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091944247000000007 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119271+0000", 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3225000000000004e-05 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119356+0000", 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5325e-05 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16012 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360154798, 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092295000000000002, 2024-09-13T04:58:31.466 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16012 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027249+0000", 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027252+0000", 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0970000000000002e-06 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027260+0000", 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2959999999999999e-06 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027267+0000", 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6059999999999999e-06 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119398+0000", 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130904 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119466+0000", 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8460000000000005e-05 2024-09-13T04:58:31.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119545+0000", 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8230000000000001e-05 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16013 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360131538999999, 2024-09-13T04:58:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092535714000000005, 2024-09-13T04:58:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.472 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16013 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027272+0000", 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027275+0000", 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1130000000000001e-06 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027284+0000", 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2430000000000005e-06 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027290+0000", 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4999999999999996e-06 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119588+0000", 2024-09-13T04:58:31.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092297660000000004 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119700+0000", 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000112526 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119809+0000", 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108262 2024-09-13T04:58:31.476 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16014 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2011136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360108477000001, 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092839985999999999, 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.477 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16014 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027295+0000", 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027298+0000", 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1329999999999999e-06 2024-09-13T04:58:31.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027307+0000", 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4859999999999997e-06 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027314+0000", 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5799999999999997e-06 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119880+0000", 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092566792999999994 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120027+0000", 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014627800000000001 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120136+0000", 2024-09-13T04:58:31.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109329 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16015 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2052096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360085062, 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093043843000000001, 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.482 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16015 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027322+0000", 2024-09-13T04:58:31.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.146e-06 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027330+0000", 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.524e-06 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027337+0000", 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4659999999999996e-06 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120194+0000", 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092856578999999995 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120268+0000", 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4470999999999999e-05 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120363+0000", 2024-09-13T04:58:31.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5253000000000004e-05 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16016 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 15.360061543, 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093219623000000001, 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:31.487 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16016 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027342+0000", 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027352+0000", 2024-09-13T04:58:31.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0000000000000001e-05 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027361+0000", 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7090000000000008e-06 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027368+0000", 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629999999999997e-06 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120393+0000", 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093025262999999997 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120471+0000", 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7688000000000006e-05 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120563+0000", 2024-09-13T04:58:31.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2003000000000007e-05 2024-09-13T04:58:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:31.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:31.492 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:31.494 INFO:teuthology.orchestra.run.smithi001.stdout:1595 write 0x1671d60 thru 0x167890a (0x6bab bytes) 2024-09-13T04:58:31.494 INFO:teuthology.orchestra.run.smithi001.stdout:1597 write 0x4068f22 thru 0x4078592 (0xf671 bytes) 2024-09-13T04:58:31.494 INFO:teuthology.orchestra.run.smithi001.stdout:1598 writesame 0xb4681b7 thru 0xb473ceb data_size 0x4b(0xbb35 bytes) 2024-09-13T04:58:31.494 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.494 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:31.495 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:31.495 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:31.496 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:31.491+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:31.496 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:31.496 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:31.496 INFO:teuthology.orchestra.run.smithi001.stdout:1601 trunc from 0xe211b99 to 0x68c76e 2024-09-13T04:58:31.497 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:31.497 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:31.497 INFO:teuthology.orchestra.run.smithi001.stdout:1602 punch from 0x2c67ec to 0x2d3403, (0xcc17 bytes) 2024-09-13T04:58:31.498 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:31.498 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:31.498 INFO:teuthology.orchestra.run.smithi001.stdout:1603 writesame 0xb3afb77 thru 0xb3bcbf7 data_size 0x1(0xd081 bytes) 2024-09-13T04:58:31.498 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.498 INFO:teuthology.orchestra.run.smithi001.stdout:1606 writesame 0x83c440f thru 0x83d0684 data_size 0x3(0xc276 bytes) 2024-09-13T04:58:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:31.503 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:1607 punch from 0xa587ff7 to 0xa589063, (0x106c bytes) 2024-09-13T04:58:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:1612 read 0x7b01898 thru 0x7b0bcf4 (0xa45d bytes) 2024-09-13T04:58:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:1613 read 0x50334f7 thru 0x5039ee1 (0x69eb bytes) 2024-09-13T04:58:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:1614 read 0x19a4065 thru 0x19ab550 (0x74ec bytes) 2024-09-13T04:58:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:1615 write 0x1c82029 thru 0x1c8acab (0x8c83 bytes) 2024-09-13T04:58:31.511 INFO:teuthology.orchestra.run.smithi001.stdout:1616 write 0x612b9c7 thru 0x612dcc2 (0x22fc bytes) 2024-09-13T04:58:31.525 INFO:teuthology.orchestra.run.smithi001.stdout:1618 write 0xbe431a1 thru 0xbe508a3 (0xd703 bytes) 2024-09-13T04:58:31.529 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:31.529 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:31.531 INFO:teuthology.orchestra.run.smithi001.stdout:1619 read 0x4045f7a thru 0x404f86a (0x98f1 bytes) 2024-09-13T04:58:31.531 INFO:teuthology.orchestra.run.smithi001.stdout:1620 read 0x7913162 thru 0x79144cb (0x136a bytes) 2024-09-13T04:58:31.532 INFO:teuthology.orchestra.run.smithi001.stdout:1621 writesame 0x1b9dd0a thru 0x1ba3f07 data_size 0x6(0x61fe bytes) 2024-09-13T04:58:31.537 INFO:teuthology.orchestra.run.smithi001.stdout:1622 write 0x2a0eee7 thru 0x2a11a4d (0x2b67 bytes) 2024-09-13T04:58:31.552 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:31.552 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:31.554 INFO:teuthology.orchestra.run.smithi001.stdout:1623 write 0xe3c979d thru 0xe3d3957 (0xa1bb bytes) 2024-09-13T04:58:31.558 INFO:teuthology.orchestra.run.smithi001.stdout:1624 read 0x5d86ac2 thru 0x5d8a300 (0x383f bytes) 2024-09-13T04:58:31.559 INFO:teuthology.orchestra.run.smithi001.stdout:1627 read 0x7c4657d thru 0x7c53c23 (0xd6a7 bytes) 2024-09-13T04:58:31.559 INFO:teuthology.orchestra.run.smithi001.stdout:1630 read 0xdbd32fe thru 0xdbd8212 (0x4f15 bytes) 2024-09-13T04:58:31.559 INFO:teuthology.orchestra.run.smithi001.stdout:1631 read 0x8cf5f76 thru 0x8cf8f31 (0x2fbc bytes) 2024-09-13T04:58:31.559 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:31.559 INFO:teuthology.orchestra.run.smithi001.stdout:1633 clone 11 order 21 su 8192 sc 6 2024-09-13T04:58:31.571 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:31.629 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:31.628+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:31.668 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:31.669 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:31.683 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:31.695 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:31.692+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:31.754 INFO:teuthology.orchestra.run.smithi001.stderr:set 3.1e pg_upmap_items mapping to [3->4,7->0] 2024-09-13T04:58:31.796 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:31.795+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:31.811 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:31.812 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:31.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:31.877 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone10 from 0xe3d3958 (overlap 0x51fa55) to 0x0 2024-09-13T04:58:31.912 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:31.911+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:31.998 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:31.998 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:31.999 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:31.995+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:32.013 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:32.019 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-13T04:58:32.022 INFO:teuthology.orchestra.run.smithi001.stdout:1634 read 0x4f349f3 thru 0x4f3ec57 (0xa265 bytes) 2024-09-13T04:58:32.023 INFO:teuthology.orchestra.run.smithi001.stdout:1635 punch from 0xdab94ce to 0xdabcfd7, (0x3b09 bytes) 2024-09-13T04:58:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:1636 trunc from 0xe3d3958 to 0xb053fc1 2024-09-13T04:58:32.060 INFO:teuthology.orchestra.run.smithi001.stdout:1637 read 0x462bdbf thru 0x4638212 (0xc454 bytes) 2024-09-13T04:58:32.060 INFO:teuthology.orchestra.run.smithi001.stdout:1638 read 0x7076dba thru 0x7082919 (0xbb60 bytes) 2024-09-13T04:58:32.063 INFO:teuthology.orchestra.run.smithi001.stdout:1639 read 0xb248ee thru 0xb33b67 (0xf27a bytes) 2024-09-13T04:58:32.074 INFO:teuthology.orchestra.run.smithi001.stdout:1640 trunc from 0xb053fc1 to 0xdd0719a 2024-09-13T04:58:32.086 INFO:teuthology.orchestra.run.smithi001.stdout:1642 read 0x246ba8e thru 0x2470c2d (0x51a0 bytes) 2024-09-13T04:58:32.086 INFO:teuthology.orchestra.run.smithi001.stdout:1643 read 0x63bb9d0 thru 0x63ca6e5 (0xed16 bytes) 2024-09-13T04:58:32.089 INFO:teuthology.orchestra.run.smithi001.stdout:1644 writesame 0x15ba218 thru 0x15c6df9 data_size 0x3(0xcbe2 bytes) 2024-09-13T04:58:32.100 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:32.099+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:32.178 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:32.179 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:32.193 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:32.199 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:32.196+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:32.290 INFO:teuthology.orchestra.run.smithi001.stdout:1645 write 0x5db0892 thru 0x5db4312 (0x3a81 bytes) 2024-09-13T04:58:32.300 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:32.296+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:32.356 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:32.356 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:32.370 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:32.420 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:32.419+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:32.464 INFO:teuthology.orchestra.run.smithi001.stdout:1646 trunc from 0xdd0719a to 0x75fdcde 2024-09-13T04:58:32.495 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:32.495 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:32.501 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:32.499+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:32.511 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:32.602 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:32.600+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:32.650 INFO:teuthology.orchestra.run.smithi001.stdout:1648 read 0x16fe3c4 thru 0x1704c4e (0x688b bytes) 2024-09-13T04:58:32.651 INFO:teuthology.orchestra.run.smithi001.stdout:1650 write 0x4a51cbe thru 0x4a548f5 (0x2c38 bytes) 2024-09-13T04:58:32.666 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:32.666 INFO:teuthology.orchestra.run.smithi001.stdout:1652 write 0x5532fd1 thru 0x554105b (0xe08b bytes) 2024-09-13T04:58:32.683 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:32.683 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:32.697 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:32.744 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:32.743+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:32.804 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:32.800+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:32.819 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:32.819 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:32.833 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:32.861 INFO:teuthology.orchestra.run.smithi001.stdout:1653 punch from 0x54a202b to 0x54ac75f, (0xa734 bytes) 2024-09-13T04:58:32.862 INFO:teuthology.orchestra.run.smithi001.stdout:1654 read 0x46cb634 thru 0x46d64cb (0xae98 bytes) 2024-09-13T04:58:32.862 INFO:teuthology.orchestra.run.smithi001.stdout:1655 write 0x901078 thru 0x906ad7 (0x5a60 bytes) 2024-09-13T04:58:32.924 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:32.923+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:32.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:58:33.006 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:33.004+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:33.006 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:33.006 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:33.020 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:33.106 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:33.103+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:33.137 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:33.137 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:33.137 INFO:teuthology.orchestra.run.smithi001.stdout:1658 write 0x375eb2e thru 0x37639b0 (0x4e83 bytes) 2024-09-13T04:58:33.158 INFO:teuthology.orchestra.run.smithi001.stdout:1659 trunc from 0x75fdcde to 0xe7a33e0 2024-09-13T04:58:33.164 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:33.165 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:33.167 INFO:teuthology.orchestra.run.smithi001.stdout:1660 write 0xadc32f6 thru 0xadc5701 (0x240c bytes) 2024-09-13T04:58:33.174 INFO:teuthology.orchestra.run.smithi001.stdout:1662 flatten 2024-09-13T04:58:33.185 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:33.229 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:33.228+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:33.309 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:33.308+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:33.329 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:33.329 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:33.345 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:33.410 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:33.407+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:33.511 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:33.507+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:33.519 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:33.519 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:33.538 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:33.589 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:33.633 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:33.632+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:33.712 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:33.708+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:33.717 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:33.717 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:33.746 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:33.764 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:58:33.787 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:33.788 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:33.810 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:33.810 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:33.814 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:33.811+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:33.828 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:33.829 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:33.845 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:33.865 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:33.865 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:33.886 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:33.896 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:33.896 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:33.896 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:33.896 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:33.897 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:33.911 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:33.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:33.914 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:33.911+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:33.933 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:33.981 INFO:teuthology.orchestra.run.smithi001.stdout:1663 read 0x66b0dc2 thru 0x66bc609 (0xb848 bytes) 2024-09-13T04:58:33.982 INFO:teuthology.orchestra.run.smithi001.stdout:1664 read 0x91ef0e2 thru 0x91fe9a0 (0xf8bf bytes) 2024-09-13T04:58:33.982 INFO:teuthology.orchestra.run.smithi001.stdout:1665 read 0x40f419e thru 0x4100d58 (0xcbbb bytes) 2024-09-13T04:58:33.982 INFO:teuthology.orchestra.run.smithi001.stdout:1666 writesame 0x755d5ef thru 0x756705e data_size 0x38(0x9a70 bytes) 2024-09-13T04:58:33.985 INFO:teuthology.orchestra.run.smithi001.stdout:1667 write 0x29053ab thru 0x2908b25 (0x377b bytes) 2024-09-13T04:58:33.989 INFO:teuthology.orchestra.run.smithi001.stdout:1668 write 0x21e6222 thru 0x21f1e29 (0xbc08 bytes) 2024-09-13T04:58:33.992 INFO:teuthology.orchestra.run.smithi001.stdout:1669 read 0xe5300ce thru 0xe53fe67 (0xfd9a bytes) 2024-09-13T04:58:33.992 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.007 INFO:teuthology.orchestra.run.smithi001.stdout:1671 trunc from 0xe7a33e0 to 0x402e106 2024-09-13T04:58:34.015 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:34.012+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:34.023 INFO:teuthology.orchestra.run.smithi001.stdout:1672 read 0x8b3831 thru 0x8bbbab (0x837b bytes) 2024-09-13T04:58:34.032 INFO:teuthology.orchestra.run.smithi001.stdout:1673 writesame 0x4e5b65a thru 0x4e666d5 data_size 0x5a(0xb07c bytes) 2024-09-13T04:58:34.032 INFO:teuthology.orchestra.run.smithi001.stdout:1675 read 0x2a80bae thru 0x2a80fc9 (0x41c bytes) 2024-09-13T04:58:34.032 INFO:teuthology.orchestra.run.smithi001.stdout:1677 punch from 0x11d4139 to 0x11d91e2, (0x50a9 bytes) 2024-09-13T04:58:34.032 INFO:teuthology.orchestra.run.smithi001.stdout:1678 write 0x4858a2d thru 0x4859b95 (0x1169 bytes) 2024-09-13T04:58:34.035 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.036 INFO:teuthology.orchestra.run.smithi001.stdout:1681 punch from 0x439c0ce to 0x43ab952, (0xf884 bytes) 2024-09-13T04:58:34.036 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.043 INFO:teuthology.orchestra.run.smithi001.stdout:1683 trunc from 0x4e666d6 to 0xa0875f1 2024-09-13T04:58:34.054 INFO:teuthology.orchestra.run.smithi001.stdout:1685 trunc from 0xa0875f1 to 0x7597970 2024-09-13T04:58:34.061 INFO:teuthology.orchestra.run.smithi001.stdout:1686 read 0x48d6427 thru 0x48d6649 (0x223 bytes) 2024-09-13T04:58:34.061 INFO:teuthology.orchestra.run.smithi001.stdout:1687 read 0x59d6c2f thru 0x59e50fb (0xe4cd bytes) 2024-09-13T04:58:34.062 INFO:teuthology.orchestra.run.smithi001.stdout:1688 punch from 0x4a3b0b3 to 0x4a3b23b, (0x188 bytes) 2024-09-13T04:58:34.062 INFO:teuthology.orchestra.run.smithi001.stdout:1689 write 0x53214e7 thru 0x532dc46 (0xc760 bytes) 2024-09-13T04:58:34.072 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:34.072 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:34.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:34.072 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:34.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.072 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16592 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.032802877000002, 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055701260000000002, 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16592 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036513+0000", 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5299999999999999e-06 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036521+0000", 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6699999999999994e-06 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036525+0000", 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3610000000000003e-06 2024-09-13T04:58:34.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092051+0000", 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05552613 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092163+0000", 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111494 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092212+0000", 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9432999999999999e-05 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16593 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.032784357000001, 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055799934000000002, 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.077 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.078 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16593 2024-09-13T04:58:34.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.078 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036532+0000", 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3810000000000002e-06 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036539+0000", 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2139999999999999e-06 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036544+0000", 2024-09-13T04:58:34.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1909999999999999e-06 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092237+0000", 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692609999999997 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092290+0000", 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3554000000000003e-05 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092329+0000", 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9335999999999997e-05 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16594 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.032765383000001, 2024-09-13T04:58:34.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055959726000000001, 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16594 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036550+0000", 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-13T04:58:34.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036557+0000", 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999999e-06 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036562+0000", 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092364+0000", 2024-09-13T04:58:34.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801515000000003 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092441+0000", 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7044000000000003e-05 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092508+0000", 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7617e-05 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16595 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.032413675000001, 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055757809999999998, 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16595 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036566+0000", 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996657 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036905+0000", 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033883999999999997 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036915+0000", 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0164e-05 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036930+0000", 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5245999999999999e-05 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092559+0000", 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055628628999999999 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092615+0000", 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6684000000000001e-05 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092658+0000", 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2559999999999999e-05 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.091 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16596 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3215360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.032176029999999, 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696469999999998, 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16596 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036936+0000", 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997983 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037142+0000", 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020614900000000001 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037152+0000", 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6739999999999992e-06 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037159+0000", 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2024-09-13T04:58:34.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092692+0000", 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055533569999999997 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092768+0000", 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5220000000000001e-05 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092834+0000", 2024-09-13T04:58:34.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6674999999999994e-05 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16597 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.032023168999999, 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055678257000000002, 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16597 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037164+0000", 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998736 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037300+0000", 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000135456 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037310+0000", 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0366e-05 2024-09-13T04:58:34.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037316+0000", 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4910000000000002e-06 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092888+0000", 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055571872000000001 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092929+0000", 2024-09-13T04:58:34.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0667999999999998e-05 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092969+0000", 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9913000000000002e-05 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16603 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.030994795000002, 2024-09-13T04:58:34.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055630449999999998, 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16603 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:34.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038321+0000", 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1179999999999999e-06 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038326+0000", 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6189999999999999e-06 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038331+0000", 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2429999999999999e-06 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093860+0000", 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055528871 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093900+0000", 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0742999999999999e-05 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093949+0000", 2024-09-13T04:58:34.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16604 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.030979166000002, 2024-09-13T04:58:34.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055794816999999997, 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16604 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038334+0000", 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038336+0000", 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.046e-06 2024-09-13T04:58:34.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038342+0000", 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038346+0000", 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1799999999999998e-06 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093983+0000", 2024-09-13T04:58:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055636906999999999 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094059+0000", 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6075999999999995e-05 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094129+0000", 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0337000000000001e-05 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16606 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.030470353999998, 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055591046999999998, 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16606 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.110 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038845+0000", 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4329999999999998e-06 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038852+0000", 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1240000000000002e-06 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038857+0000", 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9620000000000001e-06 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094307+0000", 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055450714999999998 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094392+0000", 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4586000000000006e-05 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094435+0000", 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2697000000000001e-05 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16607 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.030453216000002, 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696915999999999, 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16607 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038860+0000", 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038862+0000", 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1859999999999999e-06 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038868+0000", 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8590000000000001e-06 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038873+0000", 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6310000000000002e-06 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094459+0000", 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055585561999999998 2024-09-13T04:58:34.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094519+0000", 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0161999999999998e-05 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094558+0000", 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8831999999999997e-05 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16608 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.030436956999999, 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801370000000003, 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16608 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038879+0000", 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.21e-06 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038885+0000", 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8100000000000003e-06 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038889+0000", 2024-09-13T04:58:34.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2760000000000002e-06 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094581+0000", 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692236999999999 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094639+0000", 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7935e-05 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094678+0000", 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9267999999999997e-05 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16609 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.030207081, 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055693234000000001, 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16609 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038892+0000", 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997854 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039112+0000", 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000219402 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039125+0000", 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039135+0000", 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8549999999999997e-06 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094702+0000", 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055567083000000003 2024-09-13T04:58:34.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094761+0000", 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9425000000000001e-05 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094800+0000", 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8754999999999997e-05 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16610 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.030121974, 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0557396, 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16610 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039158+0000", 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999666 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039196+0000", 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7551e-05 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039212+0000", 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6569999999999999e-05 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039219+0000", 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5080000000000002e-06 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094824+0000", 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055604690999999998 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094890+0000", 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6204999999999994e-05 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094931+0000", 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1675000000000003e-05 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.133 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16636 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.026413165000001, 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055573821000000002, 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16636 2024-09-13T04:58:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042900+0000", 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042902+0000", 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0590000000000001e-06 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042908+0000", 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5620000000000003e-06 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042912+0000", 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.437e-06 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098321+0000", 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055408959000000001 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098401+0000", 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0122000000000003e-05 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098475+0000", 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3002000000000003e-05 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16638 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.026081716, 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055689097, 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16638 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043234+0000", 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043237+0000", 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6260000000000001e-06 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043240+0000", 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.627e-06 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098773+0000", 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055532519000000002 2024-09-13T04:58:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098849+0000", 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6808000000000001e-05 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098921+0000", 2024-09-13T04:58:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1835000000000006e-05 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16683 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.01803082, 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055570913999999999, 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16683 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051285+0000", 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051294+0000", 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2239999999999995e-06 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051299+0000", 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0259999999999998e-06 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106697+0000", 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055398672000000003 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106781+0000", 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3849000000000003e-05 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106854+0000", 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2683999999999997e-05 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16684 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.147 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.018010357000001, 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055767156999999998, 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16684 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:34.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051305+0000", 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.204e-06 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051313+0000", 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4830000000000002e-06 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051318+0000", 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6529999999999999e-06 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106896+0000", 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055577731999999998 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106987+0000", 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0920000000000004e-05 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107071+0000", 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3492999999999995e-05 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.151 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16685 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.017991208000002, 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055971087000000003, 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16685 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051322+0000", 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051325+0000", 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4499999999999998e-06 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051332+0000", 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2409999999999998e-06 2024-09-13T04:58:34.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051337+0000", 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0499999999999999e-06 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107129+0000", 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055791855000000001 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107212+0000", 2024-09-13T04:58:34.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3046000000000004e-05 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107294+0000", 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1773000000000005e-05 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16689 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.016726943999998, 2024-09-13T04:58:34.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694117000000001, 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16689 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052589+0000", 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4420000000000001e-06 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052595+0000", 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8549999999999998e-06 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052600+0000", 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8169999999999998e-06 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108111+0000", 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055511660999999997 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108201+0000", 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9386000000000001e-05 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108281+0000", 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0338000000000002e-05 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16691 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:34.160 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.016310700999998, 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055651254999999997, 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16691 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053005+0000", 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4490000000000002e-06 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053010+0000", 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2680000000000002e-06 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053014+0000", 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9729999999999999e-06 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108498+0000", 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055483769000000002 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108580+0000", 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1638999999999995e-05 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108654+0000", 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4548000000000005e-05 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.165 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.165 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:34.165 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.165 INFO:teuthology.orchestra.run.smithi001.stdout:1693 read 0x363ba60 thru 0x36474f2 (0xba93 bytes) 2024-09-13T04:58:34.165 INFO:teuthology.orchestra.run.smithi001.stdout:1694 write 0x64f45af thru 0x64ff053 (0xaaa5 bytes) 2024-09-13T04:58:34.166 INFO:teuthology.orchestra.run.smithi001.stdout:1695 punch from 0x382d14d to 0x383c1b5, (0xf068 bytes) 2024-09-13T04:58:34.166 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.166 INFO:teuthology.orchestra.run.smithi001.stdout:1697 write 0x6d20625 thru 0x6d26861 (0x623d bytes) 2024-09-13T04:58:34.167 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:34.168 INFO:teuthology.orchestra.run.smithi001.stdout:1699 write 0x8dfc99d thru 0x8e03045 (0x66a9 bytes) 2024-09-13T04:58:34.168 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.168 INFO:teuthology.orchestra.run.smithi001.stdout:1702 trunc from 0x8e03046 to 0x6f4ddaa 2024-09-13T04:58:34.168 INFO:teuthology.orchestra.run.smithi001.stdout:1703 writesame 0x4493ca0 thru 0x449bffc data_size 0x1(0x835d bytes) 2024-09-13T04:58:34.168 INFO:teuthology.orchestra.run.smithi001.stdout:1704 trunc from 0x6f4ddaa to 0x6839ba4 2024-09-13T04:58:34.168 INFO:teuthology.orchestra.run.smithi001.stdout:1706 writesame 0xe5c22d thru 0xe65909 data_size 0x1(0x96dd bytes) 2024-09-13T04:58:34.170 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:34.168+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:34.205 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.216 INFO:teuthology.orchestra.run.smithi001.stdout:1709 trunc from 0x6839ba4 to 0xd8b6d97 2024-09-13T04:58:34.225 INFO:teuthology.orchestra.run.smithi001.stdout:1710 read 0x8e8b4a1 thru 0x8e96c8a (0xb7ea bytes) 2024-09-13T04:58:34.225 INFO:teuthology.orchestra.run.smithi001.stdout:1712 read 0x4b62237 thru 0x4b67331 (0x50fb bytes) 2024-09-13T04:58:34.226 INFO:teuthology.orchestra.run.smithi001.stdout:1713 read 0xcbe72a2 thru 0xcbea085 (0x2de4 bytes) 2024-09-13T04:58:34.226 INFO:teuthology.orchestra.run.smithi001.stdout:1714 read 0x6e3ee05 thru 0x6e4b381 (0xc57d bytes) 2024-09-13T04:58:34.226 INFO:teuthology.orchestra.run.smithi001.stdout:1715 writesame 0xbfde84d thru 0xbfe2b4f data_size 0x2f(0x4303 bytes) 2024-09-13T04:58:34.233 INFO:teuthology.orchestra.run.smithi001.stdout:1716 read 0x46016a5 thru 0x4608ef9 (0x7855 bytes) 2024-09-13T04:58:34.234 INFO:teuthology.orchestra.run.smithi001.stdout:1717 punch from 0x94013ee to 0x9408da2, (0x79b4 bytes) 2024-09-13T04:58:34.235 INFO:teuthology.orchestra.run.smithi001.stdout:1718 read 0x755ced9 thru 0x75600fa (0x3222 bytes) 2024-09-13T04:58:34.235 INFO:teuthology.orchestra.run.smithi001.stdout:1719 read 0xc83764a thru 0xc83bd21 (0x46d8 bytes) 2024-09-13T04:58:34.235 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.235 INFO:teuthology.orchestra.run.smithi001.stdout:1723 write 0xa01face thru 0xa0288b8 (0x8deb bytes) 2024-09-13T04:58:34.245 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.246 INFO:teuthology.orchestra.run.smithi001.stdout:1730 write 0x32f421 thru 0x33cad0 (0xd6b0 bytes) 2024-09-13T04:58:34.255 INFO:teuthology.orchestra.run.smithi001.stdout:1731 trunc from 0xd8b6d97 to 0xb7b3238 2024-09-13T04:58:34.268 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.268 INFO:teuthology.orchestra.run.smithi001.stdout:1733 write 0x2c0cda thru 0x2cfde7 (0xf10e bytes) 2024-09-13T04:58:34.270 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:34.268+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:1734 read 0x10769fe thru 0x1082944 (0xbf47 bytes) 2024-09-13T04:58:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:1735 write 0x9d9223e thru 0x9d9b94e (0x9711 bytes) 2024-09-13T04:58:34.279 INFO:teuthology.orchestra.run.smithi001.stdout:1737 write 0x7a02a39 thru 0x7a09d21 (0x72e9 bytes) 2024-09-13T04:58:34.288 INFO:teuthology.orchestra.run.smithi001.stdout:1739 punch from 0x78b58c9 to 0x78c4952, (0xf089 bytes) 2024-09-13T04:58:34.293 INFO:teuthology.orchestra.run.smithi001.stdout:1740 trunc from 0xb7b3238 to 0xdbc1687 2024-09-13T04:58:34.300 INFO:teuthology.orchestra.run.smithi001.stdout:1742 write 0x2571356 thru 0x25799c7 (0x8672 bytes) 2024-09-13T04:58:34.306 INFO:teuthology.orchestra.run.smithi001.stdout:1743 write 0x251debb thru 0x2526ffe (0x9144 bytes) 2024-09-13T04:58:34.313 INFO:teuthology.orchestra.run.smithi001.stdout:1745 writesame 0x417e138 thru 0x4185ff6 data_size 0x1(0x7ebf bytes) 2024-09-13T04:58:34.315 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:34.315 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:34.315 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:34.332 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:34.348 INFO:teuthology.orchestra.run.smithi001.stdout:1746 writesame 0x5a53bbf thru 0x5a60d18 data_size 0x7f(0xd15a bytes) 2024-09-13T04:58:34.359 INFO:teuthology.orchestra.run.smithi001.stdout:1747 read 0xd329381 thru 0xd33448b (0xb10b bytes) 2024-09-13T04:58:34.360 INFO:teuthology.orchestra.run.smithi001.stdout:1748 read 0x6c304e1 thru 0x6c3de96 (0xd9b6 bytes) 2024-09-13T04:58:34.360 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.360 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.360 INFO:teuthology.orchestra.run.smithi001.stdout:1751 writesame 0x3daa2dc thru 0x3db93d7 data_size 0x6a(0xf0fc bytes) 2024-09-13T04:58:34.363 INFO:teuthology.orchestra.run.smithi001.stdout:1752 write 0x6b19598 thru 0x6b280ba (0xeb23 bytes) 2024-09-13T04:58:34.370 INFO:teuthology.orchestra.run.smithi001.stdout:1753 read 0x5555d07 thru 0x5561e74 (0xc16e bytes) 2024-09-13T04:58:34.370 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:34.368+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:34.371 INFO:teuthology.orchestra.run.smithi001.stdout:1754 write 0x9a38cef thru 0x9a43a91 (0xada3 bytes) 2024-09-13T04:58:34.383 INFO:teuthology.orchestra.run.smithi001.stdout:1755 write 0x7a9468e thru 0x7a9998c (0x52ff bytes) 2024-09-13T04:58:34.400 INFO:teuthology.orchestra.run.smithi001.stdout:1756 trunc from 0xdbc1687 to 0x41671d4 2024-09-13T04:58:34.432 INFO:teuthology.orchestra.run.smithi001.stdout:1757 read 0x1ceb821 thru 0x1cf8e47 (0xd627 bytes) 2024-09-13T04:58:34.434 INFO:teuthology.orchestra.run.smithi001.stdout:1758 punch from 0x3e49a94 to 0x3e55cc2, (0xc22e bytes) 2024-09-13T04:58:34.434 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.448 INFO:teuthology.orchestra.run.smithi001.stdout:1760 write 0x833a665 thru 0x8341d07 (0x76a3 bytes) 2024-09-13T04:58:34.463 INFO:teuthology.orchestra.run.smithi001.stdout:1761 trunc from 0x8341d08 to 0x48de639 2024-09-13T04:58:34.471 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:34.467+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:34.477 INFO:teuthology.orchestra.run.smithi001.stdout:1762 read 0x2c884b8 thru 0x2c90714 (0x825d bytes) 2024-09-13T04:58:34.487 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:34.487 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:34.487 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:34.487 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:34.487 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:34.493 INFO:teuthology.orchestra.run.smithi001.stdout:1764 writesame 0x8824f78 thru 0x882e39a data_size 0x1(0x9423 bytes) 2024-09-13T04:58:34.493 INFO:teuthology.orchestra.run.smithi001.stdout:1765 read 0x7c47195 thru 0x7c47dcb (0xc37 bytes) 2024-09-13T04:58:34.493 INFO:teuthology.orchestra.run.smithi001.stdout:1767 trunc from 0x882e39b to 0x863b541 2024-09-13T04:58:34.505 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:34.508 INFO:teuthology.orchestra.run.smithi001.stdout:1768 trunc from 0x863b541 to 0xd203bb5 2024-09-13T04:58:34.515 INFO:teuthology.orchestra.run.smithi001.stdout:1769 punch from 0x75d9d66 to 0x75e116d, (0x7407 bytes) 2024-09-13T04:58:34.515 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.516 INFO:teuthology.orchestra.run.smithi001.stdout:1771 writesame 0x630b706 thru 0x630c47c data_size 0x3(0xd77 bytes) 2024-09-13T04:58:34.521 INFO:teuthology.orchestra.run.smithi001.stdout:1772 writesame 0x787b408 thru 0x78827a1 data_size 0x1(0x739a bytes) 2024-09-13T04:58:34.549 INFO:teuthology.orchestra.run.smithi001.stdout:1773 write 0x8fa4768 thru 0x8fa5f2f (0x17c8 bytes) 2024-09-13T04:58:34.554 INFO:teuthology.orchestra.run.smithi001.stdout:1775 read 0x7abbb35 thru 0x7abc2f1 (0x7bd bytes) 2024-09-13T04:58:34.555 INFO:teuthology.orchestra.run.smithi001.stdout:1776 read 0x6fa92ca thru 0x6facc8a (0x39c1 bytes) 2024-09-13T04:58:34.555 INFO:teuthology.orchestra.run.smithi001.stdout:1777 writesame 0x1e880fa thru 0x1e88512 data_size 0x1(0x419 bytes) 2024-09-13T04:58:34.558 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.558 INFO:teuthology.orchestra.run.smithi001.stdout:1782 read 0x5fe5c45 thru 0x5feaf6a (0x5326 bytes) 2024-09-13T04:58:34.558 INFO:teuthology.orchestra.run.smithi001.stdout:1784 punch from 0x3730dfb to 0x3734ac6, (0x3ccb bytes) 2024-09-13T04:58:34.558 INFO:teuthology.orchestra.run.smithi001.stdout:1786 write 0x7600e1c thru 0x76027a6 (0x198b bytes) 2024-09-13T04:58:34.563 INFO:teuthology.orchestra.run.smithi001.stdout:1788 writesame 0x1c9f7b thru 0x1d0a5b data_size 0x1(0x6ae1 bytes) 2024-09-13T04:58:34.573 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:34.571+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:34.584 INFO:teuthology.orchestra.run.smithi001.stdout:1789 write 0x5300d89 thru 0x530b0b1 (0xa329 bytes) 2024-09-13T04:58:34.595 INFO:teuthology.orchestra.run.smithi001.stdout:1790 read 0xa5a5c0d thru 0xa5acfb4 (0x73a8 bytes) 2024-09-13T04:58:34.595 INFO:teuthology.orchestra.run.smithi001.stdout:1791 read 0x60da0f7 thru 0x60e5b27 (0xba31 bytes) 2024-09-13T04:58:34.604 INFO:teuthology.orchestra.run.smithi001.stdout:1793 trunc from 0xd203bb5 to 0x7bc00c8 2024-09-13T04:58:34.641 INFO:teuthology.orchestra.run.smithi001.stdout:1794 write 0xcb7a3ff thru 0xcb7fe33 (0x5a35 bytes) 2024-09-13T04:58:34.649 INFO:teuthology.orchestra.run.smithi001.stdout:1795 writesame 0x608e343 thru 0x609c85d data_size 0x1(0xe51b bytes) 2024-09-13T04:58:34.650 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.188915538, 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:58:34.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:58:34.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:37637 3.11 e41/24 3:88b77d11:::rbd_data.112a66af081a.0000000000000000:head v 41'72, mlcod=41'70)", 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.294716582, 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027255564999999999, 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355356+0000", 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356532+0000", 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001176599 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356563+0000", 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0916e-05 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356576+0000", 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3293e-05 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356700+0000", 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123779 2024-09-13T04:58:34.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356746+0000", 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6242000000000002e-05 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382519+0000", 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025772856 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382614+0000", 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4315999999999996e-05 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39466 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5537691640000002, 2024-09-13T04:58:34.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022036103000000001, 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39466 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096315+0000", 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096326+0000", 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1596999999999999e-05 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096332+0000", 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8560000000000003e-06 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112144+0000", 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015812001999999999 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112220+0000", 2024-09-13T04:58:34.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5354999999999999e-05 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.118342+0000", 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061219409999999997 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39993 3.16 3:6fe11eb9:::rbd_data.112a66af081a.0000000000000001:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131185+0000", 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5188893390000002, 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019162558, 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.667 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39993 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131185+0000", 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131185+0000", 2024-09-13T04:58:34.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131185+0000", 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131187+0000", 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0040000000000002e-06 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131195+0000", 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8650000000000001e-06 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131207+0000", 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1977999999999999e-05 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150180+0000", 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018973515999999999 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150264+0000", 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3455000000000006e-05 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150348+0000", 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4067000000000007e-05 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39999 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 376832~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5188397330000001, 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019234370000000001, 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39999 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131234+0000", 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131237+0000", 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0839999999999999e-06 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131253+0000", 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6920999999999999e-05 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131259+0000", 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.716e-06 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150375+0000", 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019115732999999999 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150416+0000", 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0791999999999998e-05 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150469+0000", 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519e-05 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40000 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 507904~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.518810438, 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019326534999999999, 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40000 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.677 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:34.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131271+0000", 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9330000000000004e-06 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131278+0000", 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9269999999999999e-06 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131282+0000", 2024-09-13T04:58:34.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8729999999999998e-06 2024-09-13T04:58:34.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150497+0000", 2024-09-13T04:58:34.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019214709 2024-09-13T04:58:34.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150537+0000", 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0197000000000002e-05 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150591+0000", 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3352e-05 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.681 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40001 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 638976~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5187873530000005, 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019422915999999998, 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40001 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131289+0000", 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.491e-06 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131306+0000", 2024-09-13T04:58:34.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6489000000000001e-05 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131311+0000", 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150619+0000", 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019307642 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150657+0000", 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8015000000000001e-05 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150710+0000", 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3109999999999998e-05 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40004 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 770048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:34.686 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5187585910000001, 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019514348000000001, 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40004 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:34.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131318+0000", 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8159999999999998e-06 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131325+0000", 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5370000000000003e-06 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131330+0000", 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6619999999999996e-06 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150741+0000", 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019410981000000001 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150778+0000", 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7793999999999998e-05 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150830+0000", 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1974000000000001e-05 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40006 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 901120~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5187406760000002, 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019616892, 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40006 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.692 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:34.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131336+0000", 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1940000000000001e-06 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131342+0000", 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0599999999999996e-06 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131346+0000", 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5569999999999997e-06 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150858+0000", 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019512057999999999 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150897+0000", 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8903999999999999e-05 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150951+0000", 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3467999999999997e-05 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40008 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1032192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.518724024, 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019721992000000001, 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40008 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131350+0000", 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131360+0000", 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5079999999999994e-06 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131367+0000", 2024-09-13T04:58:34.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8210000000000004e-06 2024-09-13T04:58:34.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131371+0000", 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8860000000000003e-06 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150979+0000", 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019607655000000002 2024-09-13T04:58:34.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151020+0000", 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0921999999999999e-05 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151073+0000", 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2567e-05 2024-09-13T04:58:34.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40010 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1163264~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5186984189999997, 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019817818000000001, 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.705 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40010 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:34.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131378+0000", 2024-09-13T04:58:34.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:58:34.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131384+0000", 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0780000000000001e-06 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131389+0000", 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5079999999999999e-06 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151101+0000", 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019712159 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151141+0000", 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0679000000000001e-05 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151194+0000", 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2589000000000001e-05 2024-09-13T04:58:34.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40012 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1294336~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5186820619999999, 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019922019999999999, 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40012 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131392+0000", 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131394+0000", 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.029e-06 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131400+0000", 2024-09-13T04:58:34.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0560000000000001e-06 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131424+0000", 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3288000000000001e-05 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151223+0000", 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019799231 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151263+0000", 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9920999999999999e-05 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151315+0000", 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1826000000000002e-05 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40014 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1425408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:34.715 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5186455260000002, 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020004087, 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40014 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:34.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131432+0000", 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9780000000000001e-06 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131441+0000", 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9590000000000001e-06 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131447+0000", 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9410000000000004e-06 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151343+0000", 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019896113999999999 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151381+0000", 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7969999999999997e-05 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151433+0000", 2024-09-13T04:58:34.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2543000000000002e-05 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40016 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1556480~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5185930890000003, 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020070607000000001, 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40016 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131481+0000", 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131484+0000", 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.326e-06 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131492+0000", 2024-09-13T04:58:34.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0139999999999998e-06 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131505+0000", 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151461+0000", 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01995535 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151500+0000", 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8760999999999997e-05 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151552+0000", 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2533e-05 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40018 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1687552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.724 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5185632650000001, 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020160419999999998, 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40018 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131514+0000", 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0740000000000001e-06 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131520+0000", 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4479999999999999e-06 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131526+0000", 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151580+0000", 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020054424000000001 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151619+0000", 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9229e-05 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151672+0000", 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2493e-05 2024-09-13T04:58:34.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40019 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 114688~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5185442919999996, 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020264445999999998, 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40019 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131532+0000", 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.367e-06 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131538+0000", 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1070000000000001e-06 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131543+0000", 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3930000000000001e-06 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151700+0000", 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020157191000000001 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151741+0000", 2024-09-13T04:58:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0864e-05 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151795+0000", 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3896e-05 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40020 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5185273910000001, 2024-09-13T04:58:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020368440000000002, 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40020 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131549+0000", 2024-09-13T04:58:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1110000000000002e-06 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131555+0000", 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3350000000000003e-06 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131560+0000", 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151823+0000", 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020262782999999999 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151863+0000", 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0151000000000002e-05 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151916+0000", 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2604e-05 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40021 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1949696~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5185105319999996, 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020470662000000001, 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40021 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131566+0000", 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.001e-06 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131571+0000", 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5110000000000003e-06 2024-09-13T04:58:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131576+0000", 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151943+0000", 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020366649000000001 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151981+0000", 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8756e-05 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152035+0000", 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3335999999999999e-05 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40023 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5184880989999998, 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020567854, 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40023 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131589+0000", 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4140000000000001e-06 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131597+0000", 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5220000000000007e-06 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131613+0000", 2024-09-13T04:58:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5628999999999999e-05 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152063+0000", 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020450032 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152101+0000", 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7886999999999998e-05 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152154+0000", 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3739999999999997e-05 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40052 3.e 3:75db92e1:::rbd_header.112a6578f5c6:head [watch reconnect cookie 139644891391936 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.5184561500000004, 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021419572000000001, 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40052 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131621+0000", 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9280000000000002e-06 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131635+0000", 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4467000000000001e-05 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131642+0000", 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.144e-06 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152171+0000", 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020529544 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152220+0000", 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.153038+0000", 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00081775299999999995 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:34.751 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:34.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:34.754 INFO:teuthology.orchestra.run.smithi001.stdout:1797 writesame 0x3a029a8 thru 0x3a0a0bf data_size 0x67(0x7718 bytes) 2024-09-13T04:58:34.754 INFO:teuthology.orchestra.run.smithi001.stdout:1798 write 0xa71d9 thru 0xb6bcd (0xf9f5 bytes) 2024-09-13T04:58:34.754 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:34.754 INFO:teuthology.orchestra.run.smithi001.stdout:1800 write 0x5f65058 thru 0x5f67cd9 (0x2c82 bytes) 2024-09-13T04:58:34.755 INFO:teuthology.orchestra.run.smithi001.stdout:1802 read 0xf3fb59 thru 0xf45b1c (0x5fc4 bytes) 2024-09-13T04:58:34.755 INFO:teuthology.orchestra.run.smithi001.stdout:1803 punch from 0xb89ddd to 0xb90de4, (0x7007 bytes) 2024-09-13T04:58:34.755 INFO:teuthology.orchestra.run.smithi001.stdout:1804 writesame 0xbbca32f thru 0xbbd3996 data_size 0x4(0x9668 bytes) 2024-09-13T04:58:34.755 INFO:teuthology.orchestra.run.smithi001.stdout:1805 writesame 0xe9db640 thru 0xe9e3445 data_size 0x39(0x7e06 bytes) 2024-09-13T04:58:34.755 INFO:teuthology.orchestra.run.smithi001.stdout:1806 read 0x2316d72 thru 0x231dc05 (0x6e94 bytes) 2024-09-13T04:58:34.755 INFO:teuthology.orchestra.run.smithi001.stdout:1807 write 0x417bfb6 thru 0x418146c (0x54b7 bytes) 2024-09-13T04:58:34.755 INFO:teuthology.orchestra.run.smithi001.stdout:1809 write 0x266d90c thru 0x2677665 (0x9d5a bytes) 2024-09-13T04:58:34.755 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:34.751+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:34.780 INFO:teuthology.orchestra.run.smithi001.stdout:1810 trunc from 0xe9e3446 to 0x3bd87c8 2024-09-13T04:58:34.814 INFO:teuthology.orchestra.run.smithi001.stdout:1811 read 0x18c2441 thru 0x18c3f44 (0x1b04 bytes) 2024-09-13T04:58:34.815 INFO:teuthology.orchestra.run.smithi001.stdout:1812 writesame 0x648e14 thru 0x653e58 data_size 0x19(0xb045 bytes) 2024-09-13T04:58:34.821 INFO:teuthology.orchestra.run.smithi001.stdout:1814 punch from 0x164673 to 0x166a11, (0x239e bytes) 2024-09-13T04:58:34.822 INFO:teuthology.orchestra.run.smithi001.stdout:1815 read 0x49712e thru 0x4a3071 (0xbf44 bytes) 2024-09-13T04:58:34.823 INFO:teuthology.orchestra.run.smithi001.stdout:1816 punch from 0x320ad06 to 0x3215470, (0xa76a bytes) 2024-09-13T04:58:34.831 INFO:teuthology.orchestra.run.smithi001.stdout:1818 write 0x43ad712 thru 0x43bbd1d (0xe60c bytes) 2024-09-13T04:58:34.842 INFO:teuthology.orchestra.run.smithi001.stdout:1820 writesame 0x1c61bbc thru 0x1c6ad4b data_size 0x44(0x9190 bytes) 2024-09-13T04:58:34.854 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:34.852+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:34.860 INFO:teuthology.orchestra.run.smithi001.stdout:1821 write 0x70f08b3 thru 0x70f6e99 (0x65e7 bytes) 2024-09-13T04:58:34.867 INFO:teuthology.orchestra.run.smithi001.stdout:1822 read 0x2aee76e thru 0x2af01f3 (0x1a86 bytes) 2024-09-13T04:58:34.867 INFO:teuthology.orchestra.run.smithi001.stdout:1823 write 0x1419bee thru 0x14284f7 (0xe90a bytes) 2024-09-13T04:58:34.874 INFO:teuthology.orchestra.run.smithi001.stdout:1826 trunc from 0x70f6e9a to 0x68931d0 2024-09-13T04:58:34.888 INFO:teuthology.orchestra.run.smithi001.stdout:1827 punch from 0x2c823f5 to 0x2c8eef4, (0xcaff bytes) 2024-09-13T04:58:34.903 INFO:teuthology.orchestra.run.smithi001.stdout:1828 write 0xc021a31 thru 0xc02735b (0x592b bytes) 2024-09-13T04:58:34.912 INFO:teuthology.orchestra.run.smithi001.stdout:1829 punch from 0x2b274f5 to 0x2b3293a, (0xb445 bytes) 2024-09-13T04:58:34.912 INFO:teuthology.orchestra.run.smithi001.stdout:1831 clone 12 order 21 su 8192 sc 6 2024-09-13T04:58:34.941 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:34.941 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:34.952 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:34.993 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:34.992+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:35.056 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:35.052+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:35.120 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:35.121 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:35.121 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:35.121 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:35.121 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:35.135 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:35.177 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:35.176+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:35.258 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:35.255+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.133240888000003, 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:58:35.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:15554 3.a 3:54a75321:::rbd_data.112af6c12117.0000000000000002:6 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.305 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:35.306 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.271872722000001, 2024-09-13T04:58:35.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047745517000000001, 2024-09-13T04:58:35.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 15554 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025346+0000", 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025348+0000", 2024-09-13T04:58:35.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1900000000000002e-06 2024-09-13T04:58:35.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025355+0000", 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7850000000000003e-06 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025360+0000", 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6789999999999996e-06 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.067347+0000", 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041987402 2024-09-13T04:58:35.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.072996+0000", 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056486540000000003 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.073092+0000", 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6151999999999997e-05 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.130800228999998, 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:58:35.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17859 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1458176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.006432350000001, 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047596271000000002, 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17859 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290789+0000", 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290794+0000", 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7610000000000004e-06 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290799+0000", 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4889999999999997e-06 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333438+0000", 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042639200000000002 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333513+0000", 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5172000000000005e-05 2024-09-13T04:58:35.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.338383+0000", 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048697929999999999 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17893 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002922937000001, 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046056773000000002, 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17893 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294298+0000", 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5000000000000002e-06 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294307+0000", 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294312+0000", 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.451e-06 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340260+0000", 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045947904999999997 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340302+0000", 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1650999999999998e-05 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340353+0000", 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1208000000000002e-05 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.324 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17894 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002902430999999, 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046155512000000003, 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17894 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294316+0000", 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294319+0000", 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2390000000000001e-06 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294325+0000", 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0340000000000002e-06 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294329+0000", 2024-09-13T04:58:35.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7509999999999999e-06 2024-09-13T04:58:35.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340378+0000", 2024-09-13T04:58:35.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046048248 2024-09-13T04:58:35.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340421+0000", 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3627000000000001e-05 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340472+0000", 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.329 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17895 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002885646999999, 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046268562999999999, 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17895 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294333+0000", 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294335+0000", 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294341+0000", 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9939999999999997e-06 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294346+0000", 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7500000000000003e-06 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340508+0000", 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046161354000000002 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340550+0000", 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2962e-05 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340602+0000", 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1665000000000001e-05 2024-09-13T04:58:35.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17896 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1912832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002868898999999, 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046367673999999998, 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17896 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294352+0000", 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.17e-06 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294358+0000", 2024-09-13T04:58:35.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2249999999999997e-06 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294363+0000", 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6990000000000002e-06 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340627+0000", 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046263836000000003 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340668+0000", 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0638e-05 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340718+0000", 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0458999999999999e-05 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17897 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1953792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:35.338 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002852090000001, 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046474117000000002, 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17897 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294369+0000", 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:58:35.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294375+0000", 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9880000000000001e-06 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294381+0000", 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7939999999999999e-06 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340744+0000", 2024-09-13T04:58:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046362962000000001 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340786+0000", 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2311000000000002e-05 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340841+0000", 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5192999999999998e-05 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17898 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1994752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002834026999999, 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046575812000000001, 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17898 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294387+0000", 2024-09-13T04:58:35.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1600000000000001e-06 2024-09-13T04:58:35.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294395+0000", 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.52e-06 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294400+0000", 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3149999999999999e-06 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340868+0000", 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046467893000000003 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340911+0000", 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3340000000000002e-05 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340961+0000", 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9934e-05 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.347 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17899 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2035712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002815368, 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691141999999998, 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17899 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294406+0000", 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294411+0000", 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294416+0000", 2024-09-13T04:58:35.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5600000000000004e-06 2024-09-13T04:58:35.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340987+0000", 2024-09-13T04:58:35.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046570645000000001 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341045+0000", 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8065000000000001e-05 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341095+0000", 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0302000000000001e-05 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17900 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2076672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002798904999999, 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046796380999999998, 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17900 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294422+0000", 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9609999999999999e-06 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294428+0000", 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9290000000000003e-06 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294432+0000", 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4190000000000004e-06 2024-09-13T04:58:35.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341123+0000", 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046690597 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341165+0000", 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2289000000000001e-05 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341217+0000", 2024-09-13T04:58:35.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1484999999999999e-05 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17901 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002782875000001, 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046918842000000002, 2024-09-13T04:58:35.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17901 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294438+0000", 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9599999999999999e-06 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294444+0000", 2024-09-13T04:58:35.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7740000000000001e-06 2024-09-13T04:58:35.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294448+0000", 2024-09-13T04:58:35.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5210000000000004e-06 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341242+0000", 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046794009999999997 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341288+0000", 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.596e-05 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341355+0000", 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6931e-05 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17904 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002766833999999, 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006963999999998, 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17904 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:35.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294454+0000", 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.979e-06 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294460+0000", 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7899999999999996e-06 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294464+0000", 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4139999999999996e-06 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341362+0000", 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046897498000000003 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341407+0000", 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4839000000000002e-05 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341459+0000", 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2719000000000001e-05 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17911 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:35.367 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002750936000002, 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047111301000000001, 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17911 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294470+0000", 2024-09-13T04:58:35.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9769999999999999e-06 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294476+0000", 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.925e-06 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294480+0000", 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5390000000000001e-06 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341487+0000", 2024-09-13T04:58:35.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006829 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341530+0000", 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2503000000000003e-05 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341580+0000", 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9848000000000001e-05 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17914 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2048000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002734883999999, 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047212684999999997, 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17914 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294486+0000", 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0899999999999999e-06 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294492+0000", 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1920000000000002e-06 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294497+0000", 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5399999999999997e-06 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341606+0000", 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047108783000000001 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341648+0000", 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1878999999999997e-05 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341697+0000", 2024-09-13T04:58:35.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9521999999999998e-05 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17916 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2088960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.002718077000001, 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047303235999999999, 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17916 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294506+0000", 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1880000000000001e-06 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294518+0000", 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1802e-05 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294523+0000", 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7219999999999999e-06 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341713+0000", 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190086999999999 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341755+0000", 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2530000000000001e-05 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341804+0000", 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9245e-05 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.841567935, 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:58:35.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:58:35.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:58:35.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.2839682879999996, 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:58:35.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:58:35.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.269457236, 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:35.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:58:35.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:58:35.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:35.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:35.397 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:35.399 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:35.399 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:35.396+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:35.500 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:35.496+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:35.531 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:35.531 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:35.531 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:35.531 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:35.545 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:35.601 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:35.600+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:35.662 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:35.662 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:35.663 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:35.663 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:35.663 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:35.676 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:35.724 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:35.723+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:35.790 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:58:35.802 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:35.799+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:35.808 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:35.808 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:35.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:35.808 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15997 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1314816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.779136771000001, 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088423697999999995, 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15997 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.809 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026558+0000", 2024-09-13T04:58:35.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026564+0000", 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4399999999999996e-06 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026580+0000", 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6149000000000001e-05 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026593+0000", 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2723e-05 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114730+0000", 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088137005000000004 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114857+0000", 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012730799999999999 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114983+0000", 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012595600000000001 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15998 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.779092243000001, 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088737302000000004, 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.813 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15998 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026603+0000", 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026608+0000", 2024-09-13T04:58:35.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7440000000000004e-06 2024-09-13T04:58:35.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026626+0000", 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8114999999999999e-05 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026638+0000", 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1976e-05 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115046+0000", 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088407311000000002 2024-09-13T04:58:35.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115180+0000", 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013447700000000001 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115341+0000", 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000160676 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.818 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15999 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.779046049000002, 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089223035000000006, 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15999 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026649+0000", 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026655+0000", 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9560000000000002e-06 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026671+0000", 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026684+0000", 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.273e-05 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115423+0000", 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088738368999999997 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115651+0000", 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022799699999999999 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115873+0000", 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222337 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16000 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.779000091, 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089511479000000005, 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16000 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.823 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026695+0000", 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026701+0000", 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9000000000000003e-06 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026717+0000", 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6379e-05 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026731+0000", 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4039000000000001e-05 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115965+0000", 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089233554000000007 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116078+0000", 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000113655 2024-09-13T04:58:35.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116207+0000", 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128942 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16001 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1478656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778907126, 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089757132000000003, 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.827 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16001 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026788+0000", 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026794+0000", 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8059999999999997e-06 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026827+0000", 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3002e-05 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026842+0000", 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4516e-05 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116278+0000", 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089435635999999999 2024-09-13T04:58:35.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116414+0000", 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000135919 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116546+0000", 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013235300000000001 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.831 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16002 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1519616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778840159000001, 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089990789000000002, 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16002 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026854+0000", 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026862+0000", 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5789999999999998e-06 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026883+0000", 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1892e-05 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026897+0000", 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.326e-05 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116609+0000", 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089712560999999996 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116708+0000", 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8787000000000001e-05 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116846+0000", 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013842999999999999 2024-09-13T04:58:35.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16003 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778787531999999, 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090223747000000007, 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.836 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16003 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026907+0000", 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026913+0000", 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026930+0000", 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7065999999999999e-05 2024-09-13T04:58:35.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026943+0000", 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3249e-05 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116903+0000", 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089959340999999998 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116992+0000", 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9690000000000004e-05 2024-09-13T04:58:35.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117132+0000", 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013984800000000001 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16004 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1601536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778728329, 2024-09-13T04:58:35.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090483628999999996, 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16004 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026967+0000", 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026978+0000", 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1219e-05 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027011+0000", 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3222000000000001e-05 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027025+0000", 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3995999999999999e-05 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117203+0000", 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090178122999999999 2024-09-13T04:58:35.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117287+0000", 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3721999999999996e-05 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117451+0000", 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016434300000000001 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16005 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778657439, 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090781484999999995, 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16005 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027037+0000", 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027044+0000", 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4640000000000003e-06 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027061+0000", 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6949e-05 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027074+0000", 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3422e-05 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117547+0000", 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.09047318 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117682+0000", 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000134602 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117820+0000", 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013803200000000001 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.848 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16006 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778608338000002, 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091107120999999999, 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16006 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027086+0000", 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027092+0000", 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027109+0000", 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6330000000000001e-05 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027119+0000", 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0132999999999999e-05 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117876+0000", 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090756823 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118024+0000", 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000148758 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118195+0000", 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017021500000000001 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16007 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1724416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.853 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778570053999999, 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091267529999999999, 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16007 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027125+0000", 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027129+0000", 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5599999999999998e-06 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027137+0000", 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7129999999999994e-06 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027144+0000", 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8329999999999996e-06 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118240+0000", 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091096166000000006 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118313+0000", 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000005e-05 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118393+0000", 2024-09-13T04:58:35.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0680999999999995e-05 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16008 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1765376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778545549, 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091580432000000003, 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.858 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16008 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027153+0000", 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3500000000000001e-06 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027161+0000", 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1799999999999996e-06 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027168+0000", 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5919999999999997e-06 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118454+0000", 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091286415999999995 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118592+0000", 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013743699999999999 2024-09-13T04:58:35.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118731+0000", 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013906300000000001 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16009 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:35.862 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778517655999998, 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091763040000000004, 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16009 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027177+0000", 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027181+0000", 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7160000000000001e-06 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027190+0000", 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4630000000000001e-06 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027196+0000", 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6429999999999998e-06 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118776+0000", 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091579259999999996 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118855+0000", 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9695000000000002e-05 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118941+0000", 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6013000000000005e-05 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.866 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16010 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1847296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778493790999999, 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091942769999999993, 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16010 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.867 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027201+0000", 2024-09-13T04:58:35.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027205+0000", 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.213e-06 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027213+0000", 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2460000000000003e-06 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027220+0000", 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9970000000000001e-06 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118986+0000", 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091766174000000006 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119056+0000", 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9988999999999997e-05 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119145+0000", 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8751000000000005e-05 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16011 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1888256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778469816000001, 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130156000000005, 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.871 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16011 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027225+0000", 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027229+0000", 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1640000000000001e-06 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027237+0000", 2024-09-13T04:58:35.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3299999999999999e-06 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027243+0000", 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4620000000000001e-06 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119188+0000", 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091944247000000007 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119271+0000", 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3225000000000004e-05 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119356+0000", 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5325e-05 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16012 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778446213999999, 2024-09-13T04:58:35.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092295000000000002, 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16012 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027249+0000", 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027252+0000", 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0970000000000002e-06 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027260+0000", 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2959999999999999e-06 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027267+0000", 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6059999999999999e-06 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119398+0000", 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130904 2024-09-13T04:58:35.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119466+0000", 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8460000000000005e-05 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119545+0000", 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8230000000000001e-05 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16013 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778422955, 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092535714000000005, 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16013 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027272+0000", 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027275+0000", 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1130000000000001e-06 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027284+0000", 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2430000000000005e-06 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027290+0000", 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4999999999999996e-06 2024-09-13T04:58:35.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119588+0000", 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092297660000000004 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119700+0000", 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000112526 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119809+0000", 2024-09-13T04:58:35.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108262 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16014 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2011136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778399893, 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092839985999999999, 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16014 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027295+0000", 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027298+0000", 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1329999999999999e-06 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027307+0000", 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4859999999999997e-06 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027314+0000", 2024-09-13T04:58:35.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5799999999999997e-06 2024-09-13T04:58:35.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119880+0000", 2024-09-13T04:58:35.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092566792999999994 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120027+0000", 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014627800000000001 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120136+0000", 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109329 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16015 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2052096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778376477999998, 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093043843000000001, 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16015 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027322+0000", 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.146e-06 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027330+0000", 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.524e-06 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027337+0000", 2024-09-13T04:58:35.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4659999999999996e-06 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120194+0000", 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092856578999999995 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120268+0000", 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4470999999999999e-05 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120363+0000", 2024-09-13T04:58:35.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5253000000000004e-05 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16016 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 19.778352958999999, 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093219623000000001, 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:35.893 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16016 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027342+0000", 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027352+0000", 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0000000000000001e-05 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027361+0000", 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7090000000000008e-06 2024-09-13T04:58:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027368+0000", 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629999999999997e-06 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120393+0000", 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093025262999999997 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120471+0000", 2024-09-13T04:58:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7688000000000006e-05 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120563+0000", 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2003000000000007e-05 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:35.897 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:35.898 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:35.940 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:35.939+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:35.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:36.004 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:36.003+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:36.010 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-13T04:58:36.012 INFO:teuthology.orchestra.run.smithi001.stdout:1833 punch from 0x8b023c to 0x8b3e69, (0x3c2d bytes) 2024-09-13T04:58:36.017 INFO:teuthology.orchestra.run.smithi001.stdout:1834 write 0x314483f thru 0x3147a06 (0x31c8 bytes) 2024-09-13T04:58:36.027 INFO:teuthology.orchestra.run.smithi001.stdout:1836 punch from 0x779c53d to 0x779e3fd, (0x1ec0 bytes) 2024-09-13T04:58:36.027 INFO:teuthology.orchestra.run.smithi001.stdout:1837 write 0xa24ac3 thru 0xa3376e (0xecac bytes) 2024-09-13T04:58:36.087 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:36.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:36.087 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:36.087 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:36.102 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:36.109 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:36.110 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:36.112 INFO:teuthology.orchestra.run.smithi001.stdout:1839 write 0xcf35d82 thru 0xcf39e42 (0x40c1 bytes) 2024-09-13T04:58:36.119 INFO:teuthology.orchestra.run.smithi001.stdout:1840 write 0xb10465c thru 0xb10e487 (0x9e2c bytes) 2024-09-13T04:58:36.132 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:36.133 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:36.142 INFO:teuthology.orchestra.run.smithi001.stdout:1841 trunc from 0xcf39e43 to 0x49ee8ab 2024-09-13T04:58:36.144 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:36.143+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:36.157 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:36.158 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:36.168 INFO:teuthology.orchestra.run.smithi001.stdout:1842 writesame 0x22799f5 thru 0x2284f70 data_size 0x73(0xb57c bytes) 2024-09-13T04:58:36.177 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:36.197 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:36.197 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:36.206 INFO:teuthology.orchestra.run.smithi001.stdout:1843 punch from 0x40ceb63 to 0x40d54fc, (0x6999 bytes) 2024-09-13T04:58:36.206 INFO:teuthology.orchestra.run.smithi001.stdout:1844 punch from 0xc435d3 to 0xc47b7c, (0x45a9 bytes) 2024-09-13T04:58:36.207 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:36.203+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:36.215 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:36.215 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:36.217 INFO:teuthology.orchestra.run.smithi001.stdout:1845 writesame 0x6d529a7 thru 0x6d583ac data_size 0x2e(0x5a06 bytes) 2024-09-13T04:58:36.218 INFO:teuthology.orchestra.run.smithi001.stdout:1846 read 0x61a26cf thru 0x61ac72e (0xa060 bytes) 2024-09-13T04:58:36.218 INFO:teuthology.orchestra.run.smithi001.stdout:1847 read 0x8359a3 thru 0x83963a (0x3c98 bytes) 2024-09-13T04:58:36.218 INFO:teuthology.orchestra.run.smithi001.stdout:1848 write 0x4caecbd thru 0x4cb5dd7 (0x711b bytes) 2024-09-13T04:58:36.221 INFO:teuthology.orchestra.run.smithi001.stdout:1849 punch from 0x5fcbd73 to 0x5fcda62, (0x1cef bytes) 2024-09-13T04:58:36.221 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:36.221 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:36.221 INFO:teuthology.orchestra.run.smithi001.stdout:1852 write 0x6750c7c thru 0x675b529 (0xa8ae bytes) 2024-09-13T04:58:36.230 INFO:teuthology.orchestra.run.smithi001.stdout:1853 write 0xf827ea thru 0xf8b5e1 (0x8df8 bytes) 2024-09-13T04:58:36.235 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:36.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:36.243 INFO:teuthology.orchestra.run.smithi001.stdout:1854 punch from 0x404c6c to 0x40ed74, (0xa108 bytes) 2024-09-13T04:58:36.244 INFO:teuthology.orchestra.run.smithi001.stdout:1856 read 0x1a68de5 thru 0x1a73b8d (0xada9 bytes) 2024-09-13T04:58:36.245 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:36.245 INFO:teuthology.orchestra.run.smithi001.stdout:1858 write 0x733831 thru 0x7434bf (0xfc8f bytes) 2024-09-13T04:58:36.256 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:36.256 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:36.259 INFO:teuthology.orchestra.run.smithi001.stdout:1859 writesame 0x93d93d9 thru 0x93e751b data_size 0x1(0xe143 bytes) 2024-09-13T04:58:36.266 INFO:teuthology.orchestra.run.smithi001.stdout:1860 write 0x97ade25 thru 0x97bcf6d (0xf149 bytes) 2024-09-13T04:58:36.274 INFO:teuthology.orchestra.run.smithi001.stdout:1861 writesame 0x585aa7b thru 0x585bf1a data_size 0x18(0x14a0 bytes) 2024-09-13T04:58:36.277 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:36.277 INFO:teuthology.orchestra.run.smithi001.stdout:1863 read 0x775c6ac thru 0x776368a (0x6fdf bytes) 2024-09-13T04:58:36.277 INFO:teuthology.orchestra.run.smithi001.stdout:1864 read 0x1a5b29 thru 0x1b01db (0xa6b3 bytes) 2024-09-13T04:58:36.279 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:36.280 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:36.288 INFO:teuthology.orchestra.run.smithi001.stdout:1865 trunc from 0x97bcf6e to 0x2a9deb7 2024-09-13T04:58:36.295 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:36.320 INFO:teuthology.orchestra.run.smithi001.stdout:1866 clone 13 order 25 su 16777216 sc 8 2024-09-13T04:58:36.336 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:36.335+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:36.408 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:36.404+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:36.456 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:36.456 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:36.470 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:36.509 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:36.508+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:36.588 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:36.588 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:36.602 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:36.610 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:36.608+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:36.711 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:36.707+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:36.726 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:36.726 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:36.740 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:36.769 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:58:36.780 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:58:36.780 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:58:36.780 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T04:58:36.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T04:58:36.828 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:36.827+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:36.869 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone12 intact 2024-09-13T04:58:36.913 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:36.911+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:36.917 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:36.917 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:36.933 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:36.945 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-13T04:58:37.020 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:37.019+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:37.103 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:37.103 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 49, 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "8850da77-4384-4335-9559-af2e805c4f72", 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-13T04:57:34.164121+0000", 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-13T04:58:36.760995+0000", 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-13T04:57:47.098606+0000", 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-13T04:57:36.815703+0000", 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5799936, 2024-09-13T04:58:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-13T04:58:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "reef", 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:48.984599+0000", 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "19", 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-13T04:58:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:52.756781+0000", 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "23", 2024-09-13T04:58:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 3, 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 23, 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 4.9800000190734863, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 4.9800000190734863, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 4.380000114440918, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 4.380000114440918, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-13T04:58:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:58:02.446076+0000", 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:58:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:58:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "49", 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:58:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 16, 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 49, 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:58:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:58:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "2fda29ca-3c6d-47ae-9244-8ccdf55f89f6", 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-13T04:58:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-13T04:58:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6820/2346968854", 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6824/2346968854", 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/2346968854", 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/2346968854", 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9ca6e867-faf0-4295-b8eb-4c738dd26cb9", 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/466532899", 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6821/466532899", 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/466532899", 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/466532899", 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d8159400-8662-4567-8714-b48c2d8f3ad2", 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:58:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/343712578", 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/343712578", 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/343712578", 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/343712578", 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:58:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "13a3ea70-fa37-41d8-bae7-0203952b02cb", 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/2021827348", 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/2021827348", 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/2021827348", 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/2021827348", 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e896e6b5-c54a-405f-ab69-2a5b7b7a153e", 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6800", 2024-09-13T04:58:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6801", 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6802", 2024-09-13T04:58:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6803", 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6806", 2024-09-13T04:58:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6807", 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6804", 2024-09-13T04:58:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6805", 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6801/2183053562", 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6803/2183053562", 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6807/2183053562", 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6805/2183053562", 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7de23072-7eda-43d6-9b92-d608ef82c6fa", 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6816", 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6817", 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6819", 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6821", 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6826", 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6828", 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6822", 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6824", 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6817/2765053006", 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6821/2765053006", 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6828/2765053006", 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6824/2765053006", 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8719fad9-81a0-46be-a546-db83bcace33b", 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 36, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6818", 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6820", 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6823", 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6825", 2024-09-13T04:58:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6830", 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6831", 2024-09-13T04:58:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6827", 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6829", 2024-09-13T04:58:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6820/2721913793", 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6825/2721913793", 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6831/2721913793", 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6829/2721913793", 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8bf0a351-e58e-4393-a6ce-fe70acf362aa", 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6808", 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6809", 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6810", 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6811", 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6814", 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6815", 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:58:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6812", 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6813", 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6809/4202513770", 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6811/4202513770", 2024-09-13T04:58:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6815/4202513770", 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6813/4202513770", 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.092681+0000", 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.414226+0000", 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.368701+0000", 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.370707+0000", 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.980085+0000", 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.874193+0000", 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:58:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.219724+0000", 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:58:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.114987+0000", 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [ 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.6", 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: 6, 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: 4 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:37.183 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:37.185 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:37.186 INFO:tasks.thrashosds.thrasher:j is {'epoch': 49, 'fsid': '8850da77-4384-4335-9559-af2e805c4f72', 'created': '2024-09-13T04:57:34.164121+0000', 'modified': '2024-09-13T04:58:36.760995+0000', 'last_up_change': '2024-09-13T04:57:47.098606+0000', 'last_in_change': '2024-09-13T04:57:36.815703+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T04:57:48.984599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T04:57:52.756781+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T04:58:02.446076+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 49, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2fda29ca-3c6d-47ae-9244-8ccdf55f89f6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6820', 'nonce': 2346968854}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6824', 'nonce': 2346968854}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 2346968854}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 2346968854}]}, 'public_addr': '172.21.15.1:6820/2346968854', 'cluster_addr': '172.21.15.1:6824/2346968854', 'heartbeat_back_addr': '172.21.15.1:6831/2346968854', 'heartbeat_front_addr': '172.21.15.1:6828/2346968854', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9ca6e867-faf0-4295-b8eb-4c738dd26cb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 466532899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6819', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 466532899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 466532899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6823', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 466532899}]}, 'public_addr': '172.21.15.1:6817/466532899', 'cluster_addr': '172.21.15.1:6821/466532899', 'heartbeat_back_addr': '172.21.15.1:6829/466532899', 'heartbeat_front_addr': '172.21.15.1:6825/466532899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8159400-8662-4567-8714-b48c2d8f3ad2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 343712578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 343712578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 343712578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 343712578}]}, 'public_addr': '172.21.15.1:6801/343712578', 'cluster_addr': '172.21.15.1:6803/343712578', 'heartbeat_back_addr': '172.21.15.1:6807/343712578', 'heartbeat_front_addr': '172.21.15.1:6805/343712578', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '13a3ea70-fa37-41d8-bae7-0203952b02cb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 2021827348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 2021827348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 2021827348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 2021827348}]}, 'public_addr': '172.21.15.1:6809/2021827348', 'cluster_addr': '172.21.15.1:6811/2021827348', 'heartbeat_back_addr': '172.21.15.1:6815/2021827348', 'heartbeat_front_addr': '172.21.15.1:6813/2021827348', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e896e6b5-c54a-405f-ab69-2a5b7b7a153e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 2183053562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 2183053562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 2183053562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 2183053562}]}, 'public_addr': '172.21.15.144:6801/2183053562', 'cluster_addr': '172.21.15.144:6803/2183053562', 'heartbeat_back_addr': '172.21.15.144:6807/2183053562', 'heartbeat_front_addr': '172.21.15.144:6805/2183053562', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7de23072-7eda-43d6-9b92-d608ef82c6fa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6816', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2765053006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 2765053006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6826', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6828', 'nonce': 2765053006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6824', 'nonce': 2765053006}]}, 'public_addr': '172.21.15.144:6817/2765053006', 'cluster_addr': '172.21.15.144:6821/2765053006', 'heartbeat_back_addr': '172.21.15.144:6828/2765053006', 'heartbeat_front_addr': '172.21.15.144:6824/2765053006', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8719fad9-81a0-46be-a546-db83bcace33b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2721913793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6823', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6825', 'nonce': 2721913793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6830', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6831', 'nonce': 2721913793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6827', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6829', 'nonce': 2721913793}]}, 'public_addr': '172.21.15.144:6820/2721913793', 'cluster_addr': '172.21.15.144:6825/2721913793', 'heartbeat_back_addr': '172.21.15.144:6831/2721913793', 'heartbeat_front_addr': '172.21.15.144:6829/2721913793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8bf0a351-e58e-4393-a6ce-fe70acf362aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6809', 'nonce': 4202513770}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6810', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 4202513770}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6815', 'nonce': 4202513770}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 4202513770}]}, 'public_addr': '172.21.15.144:6809/4202513770', 'cluster_addr': '172.21.15.144:6811/4202513770', 'heartbeat_back_addr': '172.21.15.144:6815/4202513770', 'heartbeat_front_addr': '172.21.15.144:6813/4202513770', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.092681+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.414226+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.368701+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.370707+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.980085+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.874193+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.219724+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.114987+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.6', 'osds': [6, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T04:58:37.186 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-13T04:58:37.186 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:37.179+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:37.286 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:37.283+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:37.353 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:37.353 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:37.368 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:37.384 INFO:teuthology.orchestra.run.smithi001.stdout:1868 read 0xc99b50 thru 0xca50e1 (0xb592 bytes) 2024-09-13T04:58:37.386 INFO:teuthology.orchestra.run.smithi001.stdout:1869 punch from 0x14ce03c to 0x14d5307, (0x72cb bytes) 2024-09-13T04:58:37.386 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:37.383+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:37.392 INFO:teuthology.orchestra.run.smithi001.stdout:1870 write 0x21f70b4 thru 0x2202cb1 (0xbbfe bytes) 2024-09-13T04:58:37.425 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.448 INFO:teuthology.orchestra.run.smithi001.stdout:1872 write 0xab6ad6d thru 0xab7abed (0xfe81 bytes) 2024-09-13T04:58:37.451 INFO:teuthology.orchestra.run.smithi001.stdout:1874 writesame 0x8482193 thru 0x848535d data_size 0x7(0x31cb bytes) 2024-09-13T04:58:37.488 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:37.483+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:37.558 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:37.559 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:37.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:1875 write 0x51503e9 thru 0x5150fd2 (0xbea bytes) 2024-09-13T04:58:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:1877 punch from 0x47a8272 to 0x47ae16c, (0x5efa bytes) 2024-09-13T04:58:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:1879 punch from 0x71f1c8a to 0x71fd64b, (0xb9c1 bytes) 2024-09-13T04:58:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:1880 write 0x954699 thru 0x957e05 (0x376d bytes) 2024-09-13T04:58:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:1881 trunc from 0xab7abee to 0x85d782b 2024-09-13T04:58:37.616 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:37.615+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:37.689 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:37.687+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:1882 read 0x1c424c4 thru 0x1c4ebbd (0xc6fa bytes) 2024-09-13T04:58:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:1883 write 0xb4acb3 thru 0xb52f51 (0x829f bytes) 2024-09-13T04:58:37.718 INFO:teuthology.orchestra.run.smithi001.stdout:1886 write 0xa6d799f thru 0xa6e53fa (0xda5c bytes) 2024-09-13T04:58:37.721 INFO:teuthology.orchestra.run.smithi001.stdout:1887 write 0x52b52a0 thru 0x52b74e2 (0x2243 bytes) 2024-09-13T04:58:37.723 INFO:teuthology.orchestra.run.smithi001.stdout:1888 punch from 0x528355 to 0x52ff42, (0x7bed bytes) 2024-09-13T04:58:37.723 INFO:teuthology.orchestra.run.smithi001.stdout:1890 read 0x5cfdec2 thru 0x5cff479 (0x15b8 bytes) 2024-09-13T04:58:37.723 INFO:teuthology.orchestra.run.smithi001.stdout:1891 read 0xa6944cf thru 0xa69eb3b (0xa66d bytes) 2024-09-13T04:58:37.724 INFO:teuthology.orchestra.run.smithi001.stdout:1892 read 0x908654 thru 0x9126dc (0xa089 bytes) 2024-09-13T04:58:37.726 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.726 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.726 INFO:teuthology.orchestra.run.smithi001.stdout:1897 read 0x86284ab thru 0x8631c5a (0x97b0 bytes) 2024-09-13T04:58:37.727 INFO:teuthology.orchestra.run.smithi001.stdout:1899 writesame 0x9d0f478 thru 0x9d12010 data_size 0x1(0x2b99 bytes) 2024-09-13T04:58:37.738 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:37.738 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:37.738 INFO:teuthology.orchestra.run.smithi001.stdout:1900 read 0x797d93b thru 0x798902a (0xb6f0 bytes) 2024-09-13T04:58:37.739 INFO:teuthology.orchestra.run.smithi001.stdout:1901 write 0x21bb47e thru 0x21bca31 (0x15b4 bytes) 2024-09-13T04:58:37.747 INFO:teuthology.orchestra.run.smithi001.stdout:1902 trunc from 0xa6e53fb to 0x7663dc8 2024-09-13T04:58:37.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:37.761 INFO:teuthology.orchestra.run.smithi001.stdout:1904 read 0x35091d thru 0x3592e0 (0x89c4 bytes) 2024-09-13T04:58:37.762 INFO:teuthology.orchestra.run.smithi001.stdout:1905 read 0x2974a4f thru 0x2981dbf (0xd371 bytes) 2024-09-13T04:58:37.785 INFO:teuthology.orchestra.run.smithi001.stdout:1907 write 0xdeef0f9 thru 0xdef32c6 (0x41ce bytes) 2024-09-13T04:58:37.787 INFO:teuthology.orchestra.run.smithi001.stdout:1908 write 0x46cebeb thru 0x46cece9 (0xff bytes) 2024-09-13T04:58:37.790 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:37.787+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:37.790 INFO:teuthology.orchestra.run.smithi001.stdout:1909 write 0x48c3e7b thru 0x48c6fd9 (0x315f bytes) 2024-09-13T04:58:37.792 INFO:teuthology.orchestra.run.smithi001.stdout:1911 punch from 0x60d46c8 to 0x60e2e0f, (0xe747 bytes) 2024-09-13T04:58:37.792 INFO:teuthology.orchestra.run.smithi001.stdout:1913 write 0x2d2748f thru 0x2d35a46 (0xe5b8 bytes) 2024-09-13T04:58:37.797 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.797 INFO:teuthology.orchestra.run.smithi001.stdout:1917 punch from 0x9cba1d to 0x9cc7cf, (0xdb2 bytes) 2024-09-13T04:58:37.797 INFO:teuthology.orchestra.run.smithi001.stdout:1920 punch from 0x6f5d475 to 0x6f5e466, (0xff1 bytes) 2024-09-13T04:58:37.797 INFO:teuthology.orchestra.run.smithi001.stdout:1922 read 0x4df51dc thru 0x4e04410 (0xf235 bytes) 2024-09-13T04:58:37.798 INFO:teuthology.orchestra.run.smithi001.stdout:1926 punch from 0x4dffb25 to 0x4e09625, (0x9b00 bytes) 2024-09-13T04:58:37.798 INFO:teuthology.orchestra.run.smithi001.stdout:1928 write 0xb113f50 thru 0xb1166d6 (0x2787 bytes) 2024-09-13T04:58:37.805 INFO:teuthology.orchestra.run.smithi001.stdout:1929 trunc from 0xdef32c7 to 0xb74df37 2024-09-13T04:58:37.829 INFO:teuthology.orchestra.run.smithi001.stdout:1931 write 0xc42c72b thru 0xc434cf4 (0x85ca bytes) 2024-09-13T04:58:37.843 INFO:teuthology.orchestra.run.smithi001.stdout:1932 writesame 0xc5ebd39 thru 0xc5ee391 data_size 0x1(0x2659 bytes) 2024-09-13T04:58:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:1935 writesame 0x533f05b thru 0x534bb39 data_size 0x11(0xcadf bytes) 2024-09-13T04:58:37.850 INFO:teuthology.orchestra.run.smithi001.stdout:1936 read 0x451316c thru 0x451ed19 (0xbbae bytes) 2024-09-13T04:58:37.850 INFO:teuthology.orchestra.run.smithi001.stdout:1937 read 0x1fe5817 thru 0x1feab5d (0x5347 bytes) 2024-09-13T04:58:37.851 INFO:teuthology.orchestra.run.smithi001.stdout:1938 read 0xabecb98 thru 0xabf3f50 (0x73b9 bytes) 2024-09-13T04:58:37.851 INFO:teuthology.orchestra.run.smithi001.stdout:1939 write 0x5dbeb50 thru 0x5dc4313 (0x57c4 bytes) 2024-09-13T04:58:37.855 INFO:teuthology.orchestra.run.smithi001.stdout:1940 punch from 0xacf0f99 to 0xacfe809, (0xd870 bytes) 2024-09-13T04:58:37.855 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.860 INFO:teuthology.orchestra.run.smithi001.stdout:1942 trunc from 0xc5ee392 to 0x9669c02 2024-09-13T04:58:37.891 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:37.887+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:37.891 INFO:teuthology.orchestra.run.smithi001.stdout:1943 write 0xb3d0fb3 thru 0xb3de163 (0xd1b1 bytes) 2024-09-13T04:58:37.894 INFO:teuthology.orchestra.run.smithi001.stdout:1944 punch from 0x6add873 to 0x6ae4a92, (0x721f bytes) 2024-09-13T04:58:37.894 INFO:teuthology.orchestra.run.smithi001.stdout:1946 read 0x3c07534 thru 0x3c1573c (0xe209 bytes) 2024-09-13T04:58:37.894 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.895 INFO:teuthology.orchestra.run.smithi001.stdout:1949 read 0x40c65ef thru 0x40c853c (0x1f4e bytes) 2024-09-13T04:58:37.895 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:37.895 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:37.895 INFO:teuthology.orchestra.run.smithi001.stdout:1951 write 0x951dc20 thru 0x952073b (0x2b1c bytes) 2024-09-13T04:58:37.898 INFO:teuthology.orchestra.run.smithi001.stdout:1953 punch from 0x6907fb3 to 0x6910cde, (0x8d2b bytes) 2024-09-13T04:58:37.898 INFO:teuthology.orchestra.run.smithi001.stdout:1954 read 0x971e0ec thru 0x972c07f (0xdf94 bytes) 2024-09-13T04:58:37.898 INFO:teuthology.orchestra.run.smithi001.stdout:1955 write 0x8e3f6d4 thru 0x8e48e8e (0x97bb bytes) 2024-09-13T04:58:37.905 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.905 INFO:teuthology.orchestra.run.smithi001.stdout:1957 write 0x79c3157 thru 0x79cc673 (0x951d bytes) 2024-09-13T04:58:37.911 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:37.914 INFO:teuthology.orchestra.run.smithi001.stdout:1958 read 0xaab1fa7 thru 0xaab5c33 (0x3c8d bytes) 2024-09-13T04:58:37.915 INFO:teuthology.orchestra.run.smithi001.stdout:1959 read 0x39788ba thru 0x397d77e (0x4ec5 bytes) 2024-09-13T04:58:37.916 INFO:teuthology.orchestra.run.smithi001.stdout:1960 read 0x89d37b6 thru 0x89de329 (0xab74 bytes) 2024-09-13T04:58:37.916 INFO:teuthology.orchestra.run.smithi001.stdout:1961 read 0x5168290 thru 0x5176021 (0xdd92 bytes) 2024-09-13T04:58:37.916 INFO:teuthology.orchestra.run.smithi001.stdout:1962 read 0x5cbfcf4 thru 0x5cc29df (0x2cec bytes) 2024-09-13T04:58:37.917 INFO:teuthology.orchestra.run.smithi001.stdout:1963 read 0xa5e5333 thru 0xa5e8422 (0x30f0 bytes) 2024-09-13T04:58:37.917 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.917 INFO:teuthology.orchestra.run.smithi001.stdout:1965 writesame 0x1fb45b8 thru 0x1fc3a9c data_size 0x1(0xf4e5 bytes) 2024-09-13T04:58:37.978 INFO:teuthology.orchestra.run.smithi001.stdout:1966 write 0xdd053a1 thru 0xdd11960 (0xc5c0 bytes) 2024-09-13T04:58:37.981 INFO:teuthology.orchestra.run.smithi001.stdout:1967 punch from 0xaf9cb50 to 0xafab188, (0xe638 bytes) 2024-09-13T04:58:37.981 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:37.982 INFO:teuthology.orchestra.run.smithi001.stdout:1969 write 0x86563ea thru 0x8663b26 (0xd73d bytes) 2024-09-13T04:58:37.990 INFO:teuthology.orchestra.run.smithi001.stdout:1970 write 0x200d520 thru 0x200e5ce (0x10af bytes) 2024-09-13T04:58:37.992 INFO:teuthology.orchestra.run.smithi001.stdout:1972 read 0x3ce244e thru 0x3ce34eb (0x109e bytes) 2024-09-13T04:58:37.992 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:37.991+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:37.992 INFO:teuthology.orchestra.run.smithi001.stdout:1973 writesame 0x4307b41 thru 0x4316e26 data_size 0x1(0xf2e6 bytes) 2024-09-13T04:58:38.037 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:38.038 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:38.052 INFO:teuthology.orchestra.run.smithi001.stdout:1974 write 0xc89bf5e thru 0xc8a56dd (0x9780 bytes) 2024-09-13T04:58:38.053 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:38.060 INFO:teuthology.orchestra.run.smithi001.stdout:1976 write 0x93ed39e thru 0x93eee4a (0x1aad bytes) 2024-09-13T04:58:38.063 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.063 INFO:teuthology.orchestra.run.smithi001.stdout:1978 writesame 0x25abb87 thru 0x25b1eff data_size 0xb(0x6379 bytes) 2024-09-13T04:58:38.068 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.068 INFO:teuthology.orchestra.run.smithi001.stdout:1982 writesame 0xb9fde08 thru 0xba0cd15 data_size 0x25(0xef0e bytes) 2024-09-13T04:58:38.072 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.072 INFO:teuthology.orchestra.run.smithi001.stdout:1984 read 0xd5624f4 thru 0xd56600a (0x3b17 bytes) 2024-09-13T04:58:38.073 INFO:teuthology.orchestra.run.smithi001.stdout:1985 write 0x779163f thru 0x7791a82 (0x444 bytes) 2024-09-13T04:58:38.075 INFO:teuthology.orchestra.run.smithi001.stdout:1988 read 0xceb3ff9 thru 0xcebdd8f (0x9d97 bytes) 2024-09-13T04:58:38.076 INFO:teuthology.orchestra.run.smithi001.stdout:1989 writesame 0x36afdaf thru 0x36ba986 data_size 0x8d(0xabd8 bytes) 2024-09-13T04:58:38.078 INFO:teuthology.orchestra.run.smithi001.stdout:1990 writesame 0xb3f4557 thru 0xb3fff69 data_size 0x7(0xba13 bytes) 2024-09-13T04:58:38.084 INFO:teuthology.orchestra.run.smithi001.stdout:1991 write 0x2cbe540 thru 0x2cca203 (0xbcc4 bytes) 2024-09-13T04:58:38.093 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:38.087+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:38.094 INFO:teuthology.orchestra.run.smithi001.stdout:1992 trunc from 0xdd11961 to 0xb70577d 2024-09-13T04:58:38.112 INFO:teuthology.orchestra.run.smithi001.stdout:1993 punch from 0x70a5851 to 0x70b51c3, (0xf972 bytes) 2024-09-13T04:58:38.112 INFO:teuthology.orchestra.run.smithi001.stdout:1994 read 0xb63fee7 thru 0xb648472 (0x858c bytes) 2024-09-13T04:58:38.113 INFO:teuthology.orchestra.run.smithi001.stdout:1995 punch from 0x41e350a to 0x41e4a58, (0x154e bytes) 2024-09-13T04:58:38.113 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.118 INFO:teuthology.orchestra.run.smithi001.stdout:1999 trunc from 0xb70577d to 0xe9182d9 2024-09-13T04:58:38.129 INFO:teuthology.orchestra.run.smithi001.stdout:2000 write 0x23afb3e thru 0x23b20b3 (0x2576 bytes) 2024-09-13T04:58:38.131 INFO:teuthology.orchestra.run.smithi001.stdout:2001 read 0x6b77cf2 thru 0x6b78641 (0x950 bytes) 2024-09-13T04:58:38.131 INFO:teuthology.orchestra.run.smithi001.stdout:2002 punch from 0x6ee9f58 to 0x6ef4545, (0xa5ed bytes) 2024-09-13T04:58:38.148 INFO:teuthology.orchestra.run.smithi001.stdout:2005 trunc from 0xe9182d9 to 0x41ea327 2024-09-13T04:58:38.168 INFO:teuthology.orchestra.run.smithi001.stdout:2006 write 0x15cb7e9 thru 0x15cf234 (0x3a4c bytes) 2024-09-13T04:58:38.183 INFO:teuthology.orchestra.run.smithi001.stdout:2008 trunc from 0x41ea327 to 0xa6ffa1b 2024-09-13T04:58:38.193 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:38.191+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:38.194 INFO:teuthology.orchestra.run.smithi001.stdout:2009 write 0x859afc5 thru 0x85aa9be (0xf9fa bytes) 2024-09-13T04:58:38.203 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.203 INFO:teuthology.orchestra.run.smithi001.stdout:2013 read 0x6ae555b thru 0x6ae997c (0x4422 bytes) 2024-09-13T04:58:38.203 INFO:teuthology.orchestra.run.smithi001.stdout:2016 write 0x20ab58c thru 0x20b4c29 (0x969e bytes) 2024-09-13T04:58:38.207 INFO:teuthology.orchestra.run.smithi001.stdout:2017 write 0x5e9ddc2 thru 0x5ea478a (0x69c9 bytes) 2024-09-13T04:58:38.211 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:38.211 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:38.215 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.215 INFO:teuthology.orchestra.run.smithi001.stdout:2022 write 0x553beed thru 0x5545e7a (0x9f8e bytes) 2024-09-13T04:58:38.220 INFO:teuthology.orchestra.run.smithi001.stdout:2023 write 0xa175bc0 thru 0xa17d209 (0x764a bytes) 2024-09-13T04:58:38.224 INFO:teuthology.orchestra.run.smithi001.stdout:2024 flatten 2024-09-13T04:58:38.227 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:38.237 INFO:teuthology.orchestra.run.smithi001.stdout:2027 write 0x2da3914 thru 0x2daac36 (0x7323 bytes) 2024-09-13T04:58:38.240 INFO:teuthology.orchestra.run.smithi001.stdout:2028 read 0x730ca90 thru 0x730d181 (0x6f2 bytes) 2024-09-13T04:58:38.240 INFO:teuthology.orchestra.run.smithi001.stdout:2030 read 0x99d96cb thru 0x99d97da (0x110 bytes) 2024-09-13T04:58:38.241 INFO:teuthology.orchestra.run.smithi001.stdout:2031 punch from 0x5f4fb34 to 0x5f5f96c, (0xfe38 bytes) 2024-09-13T04:58:38.241 INFO:teuthology.orchestra.run.smithi001.stdout:2033 punch from 0x35860cb to 0x3590ae5, (0xaa1a bytes) 2024-09-13T04:58:38.241 INFO:teuthology.orchestra.run.smithi001.stdout:2035 write 0x281f2d6 thru 0x281f7b0 (0x4db bytes) 2024-09-13T04:58:38.251 INFO:teuthology.orchestra.run.smithi001.stdout:2036 trunc from 0xa6ffa1b to 0x52b7319 2024-09-13T04:58:38.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:38.281 INFO:teuthology.orchestra.run.smithi001.stdout:2037 write 0x6adc282 thru 0x6ae9e1c (0xdb9b bytes) 2024-09-13T04:58:38.285 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.285 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.291 INFO:teuthology.orchestra.run.smithi001.stdout:2042 trunc from 0x6ae9e1d to 0xa89c813 2024-09-13T04:58:38.294 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:38.291+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:38.301 INFO:teuthology.orchestra.run.smithi001.stdout:2043 read 0x649a911 thru 0x649ddb7 (0x34a7 bytes) 2024-09-13T04:58:38.302 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.302 INFO:teuthology.orchestra.run.smithi001.stdout:2046 punch from 0x44fdaa4 to 0x450696b, (0x8ec7 bytes) 2024-09-13T04:58:38.302 INFO:teuthology.orchestra.run.smithi001.stdout:2047 read 0x5cab3c1 thru 0x5cbadfc (0xfa3c bytes) 2024-09-13T04:58:38.302 INFO:teuthology.orchestra.run.smithi001.stdout:2048 read 0x8f58354 thru 0x8f6254d (0xa1fa bytes) 2024-09-13T04:58:38.316 INFO:teuthology.orchestra.run.smithi001.stdout:2050 trunc from 0xa89c813 to 0x2c93600 2024-09-13T04:58:38.349 INFO:teuthology.orchestra.run.smithi001.stdout:2051 writesame 0x3de4ac6 thru 0x3de8609 data_size 0x2(0x3b44 bytes) 2024-09-13T04:58:38.358 INFO:teuthology.orchestra.run.smithi001.stdout:2052 write 0x415fec9 thru 0x4161c60 (0x1d98 bytes) 2024-09-13T04:58:38.362 INFO:teuthology.orchestra.run.smithi001.stdout:2054 read 0x1ee6a02 thru 0x1eeb06b (0x466a bytes) 2024-09-13T04:58:38.362 INFO:teuthology.orchestra.run.smithi001.stdout:2055 read 0x3dd762d thru 0x3de6d83 (0xf757 bytes) 2024-09-13T04:58:38.362 INFO:teuthology.orchestra.run.smithi001.stdout:2056 read 0x16c6655 thru 0x16d004b (0x99f7 bytes) 2024-09-13T04:58:38.363 INFO:teuthology.orchestra.run.smithi001.stdout:2057 write 0x2d28707 thru 0x2d28d3b (0x635 bytes) 2024-09-13T04:58:38.364 INFO:teuthology.orchestra.run.smithi001.stdout:2058 read 0x22a8b73 thru 0x22b2914 (0x9da2 bytes) 2024-09-13T04:58:38.364 INFO:teuthology.orchestra.run.smithi001.stdout:2061 read 0x1a52a5b thru 0x1a5642a (0x39d0 bytes) 2024-09-13T04:58:38.384 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:38.384 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:38.387 INFO:teuthology.orchestra.run.smithi001.stdout:2062 write 0xc61c4ff thru 0xc627acc (0xb5ce bytes) 2024-09-13T04:58:38.390 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.390 INFO:teuthology.orchestra.run.smithi001.stdout:2064 punch from 0x53b3ced to 0x53c141d, (0xd730 bytes) 2024-09-13T04:58:38.390 INFO:teuthology.orchestra.run.smithi001.stdout:2065 write 0x7406abb thru 0x740bd65 (0x52ab bytes) 2024-09-13T04:58:38.395 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:38.391+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:38.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:38.409 INFO:teuthology.orchestra.run.smithi001.stdout:2068 trunc from 0xc627acd to 0x3c50bb6 2024-09-13T04:58:38.441 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:38.441 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:38.442 INFO:teuthology.orchestra.run.smithi001.stdout:2069 write 0x9092cfd thru 0x9098e45 (0x6149 bytes) 2024-09-13T04:58:38.457 INFO:teuthology.orchestra.run.smithi001.stdout:2070 writesame 0xb4e3ccd thru 0xb4efa1c data_size 0x68(0xbd50 bytes) 2024-09-13T04:58:38.457 INFO:teuthology.orchestra.run.smithi001.stdout:2072 read 0xa68aa68 thru 0xa68e5cf (0x3b68 bytes) 2024-09-13T04:58:38.457 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.457 INFO:teuthology.orchestra.run.smithi001.stdout:2076 writesame 0x892a770 thru 0x89371c5 data_size 0x61(0xca56 bytes) 2024-09-13T04:58:38.460 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:38.460 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:38.461 INFO:teuthology.orchestra.run.smithi001.stdout:2077 punch from 0x571aa57 to 0x571adb0, (0x359 bytes) 2024-09-13T04:58:38.461 INFO:teuthology.orchestra.run.smithi001.stdout:2079 punch from 0x8cdf23f to 0x8ce346c, (0x422d bytes) 2024-09-13T04:58:38.462 INFO:teuthology.orchestra.run.smithi001.stdout:2080 write 0x9fb6e51 thru 0x9fbea65 (0x7c15 bytes) 2024-09-13T04:58:38.464 INFO:teuthology.orchestra.run.smithi001.stdout:2081 punch from 0x742d3d4 to 0x742f2be, (0x1eea bytes) 2024-09-13T04:58:38.467 INFO:teuthology.orchestra.run.smithi001.stdout:2084 trunc from 0xb4efa1d to 0xd3fc4be 2024-09-13T04:58:38.479 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:38.479 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:38.492 INFO:teuthology.orchestra.run.smithi001.stdout:2085 trunc from 0xd3fc4be to 0x23d7e82 2024-09-13T04:58:38.496 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:38.491+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:38.498 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:38.498 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:38.519 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:38.519 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:38.526 INFO:teuthology.orchestra.run.smithi001.stdout:2090 write 0xaef2b54 thru 0xaefba73 (0x8f20 bytes) 2024-09-13T04:58:38.527 INFO:teuthology.orchestra.run.smithi001.stdout:2091 writesame 0x30b065b thru 0x30bd41e data_size 0x1a(0xcdc4 bytes) 2024-09-13T04:58:38.532 INFO:teuthology.orchestra.run.smithi001.stdout:2092 read 0xa772bb6 thru 0xa780873 (0xdcbe bytes) 2024-09-13T04:58:38.542 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:38.543 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:38.543 INFO:teuthology.orchestra.run.smithi001.stdout:2093 trunc from 0xaefba74 to 0x46ee017 2024-09-13T04:58:38.545 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16592 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.50779318, 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055701260000000002, 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16592 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036511+0000", 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036513+0000", 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5299999999999999e-06 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036521+0000", 2024-09-13T04:58:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6699999999999994e-06 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036525+0000", 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3610000000000003e-06 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092051+0000", 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05552613 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092163+0000", 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111494 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092212+0000", 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9432999999999999e-05 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16593 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3661824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.507774659999999, 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055799934000000002, 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16593 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036529+0000", 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036532+0000", 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3810000000000002e-06 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036539+0000", 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2139999999999999e-06 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036544+0000", 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1909999999999999e-06 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092237+0000", 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692609999999997 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092290+0000", 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3554000000000003e-05 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092329+0000", 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9335999999999997e-05 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16594 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.507755685999999, 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055959726000000001, 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16594 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036548+0000", 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036550+0000", 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2019999999999998e-06 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036557+0000", 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999999e-06 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036562+0000", 2024-09-13T04:58:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092364+0000", 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801515000000003 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092441+0000", 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7044000000000003e-05 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092508+0000", 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7617e-05 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16595 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.507403977999999, 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055757809999999998, 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16595 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036900+0000", 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036566+0000", 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996657 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036905+0000", 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033883999999999997 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036915+0000", 2024-09-13T04:58:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0164e-05 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036930+0000", 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5245999999999999e-05 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092559+0000", 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055628628999999999 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092615+0000", 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6684000000000001e-05 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092658+0000", 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2559999999999999e-05 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16596 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3215360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.507166333000001, 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696469999999998, 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16596 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037138+0000", 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.036936+0000", 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997983 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037142+0000", 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020614900000000001 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037152+0000", 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6739999999999992e-06 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037159+0000", 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2024-09-13T04:58:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092692+0000", 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055533569999999997 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092768+0000", 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5220000000000001e-05 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092834+0000", 2024-09-13T04:58:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6674999999999994e-05 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16597 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.507013472000001, 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055678257000000002, 2024-09-13T04:58:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16597 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037291+0000", 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037164+0000", 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998736 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037300+0000", 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000135456 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037310+0000", 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0366e-05 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.037316+0000", 2024-09-13T04:58:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4910000000000002e-06 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092888+0000", 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055571872000000001 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092929+0000", 2024-09-13T04:58:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0667999999999998e-05 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.092969+0000", 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9913000000000002e-05 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16603 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3379200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.505985098, 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055630449999999998, 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16603 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038319+0000", 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038321+0000", 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1179999999999999e-06 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038326+0000", 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6189999999999999e-06 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038331+0000", 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2429999999999999e-06 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093860+0000", 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055528871 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093900+0000", 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0742999999999999e-05 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093949+0000", 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16604 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.505969469, 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055794816999999997, 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16604 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038335+0000", 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038334+0000", 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038336+0000", 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.046e-06 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038342+0000", 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038346+0000", 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1799999999999998e-06 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.093983+0000", 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055636906999999999 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094059+0000", 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6075999999999995e-05 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094129+0000", 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0337000000000001e-05 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16606 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3948544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.505460657, 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055591046999999998, 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16606 2024-09-13T04:58:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038843+0000", 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038845+0000", 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4329999999999998e-06 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038852+0000", 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1240000000000002e-06 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038857+0000", 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9620000000000001e-06 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094307+0000", 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055450714999999998 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094392+0000", 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4586000000000006e-05 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094435+0000", 2024-09-13T04:58:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2697000000000001e-05 2024-09-13T04:58:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16607 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3461120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.505443519, 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055696915999999999, 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16607 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038861+0000", 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038860+0000", 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038862+0000", 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1859999999999999e-06 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038868+0000", 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8590000000000001e-06 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038873+0000", 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6310000000000002e-06 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094459+0000", 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055585561999999998 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094519+0000", 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0161999999999998e-05 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094558+0000", 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8831999999999997e-05 2024-09-13T04:58:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16608 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.505427260000001, 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055801370000000003, 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16608 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038877+0000", 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038879+0000", 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.21e-06 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038885+0000", 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8100000000000003e-06 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038889+0000", 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2760000000000002e-06 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094581+0000", 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055692236999999999 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094639+0000", 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7935e-05 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094678+0000", 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9267999999999997e-05 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16609 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.505197383999999, 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055693234000000001, 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16609 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039107+0000", 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.038892+0000", 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997854 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039112+0000", 2024-09-13T04:58:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000219402 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039125+0000", 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039135+0000", 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8549999999999997e-06 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094702+0000", 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055567083000000003 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094761+0000", 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9425000000000001e-05 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094800+0000", 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8754999999999997e-05 2024-09-13T04:58:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16610 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.505112276999998, 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0557396, 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16610 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039192+0000", 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039158+0000", 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999666 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039196+0000", 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7551e-05 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039212+0000", 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6569999999999999e-05 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.039219+0000", 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5080000000000002e-06 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094824+0000", 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055604690999999998 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094890+0000", 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6204999999999994e-05 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.094931+0000", 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1675000000000003e-05 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16636 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.501403467999999, 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055573821000000002, 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16636 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042901+0000", 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042900+0000", 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042902+0000", 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0590000000000001e-06 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042908+0000", 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5620000000000003e-06 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.042912+0000", 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.437e-06 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098321+0000", 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055408959000000001 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098401+0000", 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0122000000000003e-05 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098475+0000", 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3002000000000003e-05 2024-09-13T04:58:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16638 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.501072018999999, 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055689097, 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16638 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043232+0000", 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043234+0000", 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043237+0000", 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6260000000000001e-06 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.043240+0000", 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.627e-06 2024-09-13T04:58:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098773+0000", 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055532519000000002 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098849+0000", 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6808000000000001e-05 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.098921+0000", 2024-09-13T04:58:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1835000000000006e-05 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16683 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3514368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.493021122999998, 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055570913999999999, 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16683 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051283+0000", 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051285+0000", 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8729999999999999e-06 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051294+0000", 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2239999999999995e-06 2024-09-13T04:58:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051299+0000", 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0259999999999998e-06 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106697+0000", 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055398672000000003 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106781+0000", 2024-09-13T04:58:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3849000000000003e-05 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106854+0000", 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2683999999999997e-05 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16684 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3555328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.49300066, 2024-09-13T04:58:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055767156999999998, 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16684 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051303+0000", 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051305+0000", 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.204e-06 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051313+0000", 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4830000000000002e-06 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051318+0000", 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6529999999999999e-06 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106896+0000", 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055577731999999998 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.106987+0000", 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0920000000000004e-05 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107071+0000", 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3492999999999995e-05 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16685 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.492981511, 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055971087000000003, 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16685 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051323+0000", 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051322+0000", 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051325+0000", 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4499999999999998e-06 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051332+0000", 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2409999999999998e-06 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.051337+0000", 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0499999999999999e-06 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107129+0000", 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055791855000000001 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107212+0000", 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3046000000000004e-05 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.107294+0000", 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1773000000000005e-05 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16689 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3760128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.491717247, 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694117000000001, 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16689 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052587+0000", 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052589+0000", 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4420000000000001e-06 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052595+0000", 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8549999999999998e-06 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.052600+0000", 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8169999999999998e-06 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108111+0000", 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055511660999999997 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108201+0000", 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9386000000000001e-05 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108281+0000", 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0338000000000002e-05 2024-09-13T04:58:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:16691 3.10 3:0b7a19ef:::rbd_data.112af6c12117.0000000000000005:6 [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.491301004, 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055651254999999997, 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 16691 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053003+0000", 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053005+0000", 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4490000000000002e-06 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053010+0000", 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2680000000000002e-06 2024-09-13T04:58:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.053014+0000", 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9729999999999999e-06 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108498+0000", 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055483769000000002 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108580+0000", 2024-09-13T04:58:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1638999999999995e-05 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.108654+0000", 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4548000000000005e-05 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:38.640 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:38.642 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.642 INFO:teuthology.orchestra.run.smithi001.stdout:2095 read 0x4033990 thru 0x4041b1a (0xe18b bytes) 2024-09-13T04:58:38.642 INFO:teuthology.orchestra.run.smithi001.stdout:2096 read 0x3ebdd58 thru 0x3ecd537 (0xf7e0 bytes) 2024-09-13T04:58:38.642 INFO:teuthology.orchestra.run.smithi001.stdout:2098 writesame 0x149457f thru 0x149af56 data_size 0xc(0x69d8 bytes) 2024-09-13T04:58:38.642 INFO:teuthology.orchestra.run.smithi001.stdout:2099 read 0x15931a3 thru 0x15a10b4 (0xdf12 bytes) 2024-09-13T04:58:38.643 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:38.643 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:38.644 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:38.644 INFO:teuthology.orchestra.run.smithi001.stdout:2100 write 0xd1f90d9 thru 0xd207ce4 (0xec0c bytes) 2024-09-13T04:58:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:2101 punch from 0x82af04 to 0x832a13, (0x7b0f bytes) 2024-09-13T04:58:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:2103 read 0x15ce17e thru 0x15d601e (0x7ea1 bytes) 2024-09-13T04:58:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:2104 punch from 0x3ed3857 to 0x3ede3e2, (0xab8b bytes) 2024-09-13T04:58:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:2105 write 0x7a759fb thru 0x7a85002 (0xf608 bytes) 2024-09-13T04:58:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:38.646 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:2110 writesame 0xc85c699 thru 0xc85f85a data_size 0x21(0x31c2 bytes) 2024-09-13T04:58:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:2111 write 0x3033983 thru 0x303a812 (0x6e90 bytes) 2024-09-13T04:58:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:2113 read 0xb868824 thru 0xb87039f (0x7b7c bytes) 2024-09-13T04:58:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:2115 read 0x3d0bc66 thru 0x3d0e8ae (0x2c49 bytes) 2024-09-13T04:58:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:2118 read 0x2fe2dcc thru 0x2fe60f5 (0x332a bytes) 2024-09-13T04:58:38.648 INFO:teuthology.orchestra.run.smithi001.stdout:2120 trunc from 0xd207ce5 to 0x5a2ebc9 2024-09-13T04:58:38.648 INFO:teuthology.orchestra.run.smithi001.stdout:2121 punch from 0x320c70a to 0x3213b8c, (0x7482 bytes) 2024-09-13T04:58:38.648 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.648 INFO:teuthology.orchestra.run.smithi001.stdout:2123 write 0x3691b4d thru 0x3694f1d (0x33d1 bytes) 2024-09-13T04:58:38.649 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:38.639+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:38.656 INFO:teuthology.orchestra.run.smithi001.stdout:2126 writesame 0xa9063d1 thru 0xa90746a data_size 0xa(0x109a bytes) 2024-09-13T04:58:38.674 INFO:teuthology.orchestra.run.smithi001.stdout:2127 write 0xdd9b635 thru 0xdd9c17c (0xb48 bytes) 2024-09-13T04:58:38.675 INFO:teuthology.orchestra.run.smithi001.stdout:2128 writesame 0xaef84a7 thru 0xaefa14e data_size 0x1c(0x1ca8 bytes) 2024-09-13T04:58:38.677 INFO:teuthology.orchestra.run.smithi001.stdout:2129 write 0x87f4638 thru 0x87fac00 (0x65c9 bytes) 2024-09-13T04:58:38.682 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:38.682 INFO:teuthology.orchestra.run.smithi001.stdout:2133 read 0xd41780 thru 0xd478de (0x615f bytes) 2024-09-13T04:58:38.683 INFO:teuthology.orchestra.run.smithi001.stdout:2136 read 0x6c94522 thru 0x6c98d8c (0x486b bytes) 2024-09-13T04:58:38.683 INFO:teuthology.orchestra.run.smithi001.stdout:2138 write 0xbd488e8 thru 0xbd5768c (0xeda5 bytes) 2024-09-13T04:58:38.687 INFO:teuthology.orchestra.run.smithi001.stdout:2140 write 0x6aa2bda thru 0x6aa9952 (0x6d79 bytes) 2024-09-13T04:58:38.694 INFO:teuthology.orchestra.run.smithi001.stdout:2143 read 0x556ce68 thru 0x55798c2 (0xca5b bytes) 2024-09-13T04:58:38.694 INFO:teuthology.orchestra.run.smithi001.stdout:2144 read 0x968f778 thru 0x969ad7c (0xb605 bytes) 2024-09-13T04:58:38.694 INFO:teuthology.orchestra.run.smithi001.stdout:2148 read 0x108da12 thru 0x109ba97 (0xe086 bytes) 2024-09-13T04:58:38.710 INFO:teuthology.orchestra.run.smithi001.stdout:2150 trunc from 0xdd9c17d to 0x41ecd6a 2024-09-13T04:58:38.735 INFO:teuthology.orchestra.run.smithi001.stdout:2151 trunc from 0x41ecd6a to 0x751f987 2024-09-13T04:58:38.742 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:38.739+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:2152 punch from 0x8acb77 to 0x8ae6b6, (0x1b3f bytes) 2024-09-13T04:58:38.751 INFO:teuthology.orchestra.run.smithi001.stdout:2153 trunc from 0x751f987 to 0xa22f218 2024-09-13T04:58:38.763 INFO:teuthology.orchestra.run.smithi001.stdout:2154 write 0x668e359 thru 0x6699fc1 (0xbc69 bytes) 2024-09-13T04:58:38.772 INFO:teuthology.orchestra.run.smithi001.stdout:2155 clone 14 order 18 su 32768 sc 12 2024-09-13T04:58:38.785 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:38.785 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:38.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:58:38.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:38.843 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:38.839+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:38.944 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:38.939+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:38.993 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:38.993 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:39.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:39.033 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-13T04:58:39.077 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:39.075+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:39.130 INFO:teuthology.orchestra.run.smithi001.stdout:2156 trunc from 0xa22f218 to 0x60ef1dd 2024-09-13T04:58:39.146 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:39.143+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.702214281, 2024-09-13T04:58:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:58:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:58:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:37637 3.11 e41/24 3:88b77d11:::rbd_data.112a66af081a.0000000000000000:head v 41'72, mlcod=41'70)", 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.808015325, 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027255564999999999, 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355356+0000", 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356532+0000", 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001176599 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356563+0000", 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0916e-05 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356576+0000", 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3293e-05 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356700+0000", 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123779 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356746+0000", 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6242000000000002e-05 2024-09-13T04:58:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382519+0000", 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025772856 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382614+0000", 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4315999999999996e-05 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39466 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.067067907, 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022036103000000001, 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39466 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096315+0000", 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096326+0000", 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1596999999999999e-05 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096332+0000", 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8560000000000003e-06 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112144+0000", 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015812001999999999 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112220+0000", 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5354999999999999e-05 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.118342+0000", 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061219409999999997 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39999 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 376832~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.032138476, 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019234370000000001, 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39999 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131234+0000", 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131237+0000", 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0839999999999999e-06 2024-09-13T04:58:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131253+0000", 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6920999999999999e-05 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131259+0000", 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.716e-06 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150375+0000", 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019115732999999999 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150416+0000", 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0791999999999998e-05 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150469+0000", 2024-09-13T04:58:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519e-05 2024-09-13T04:58:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40000 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 507904~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.032109180999999, 2024-09-13T04:58:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019326534999999999, 2024-09-13T04:58:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40000 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131271+0000", 2024-09-13T04:58:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9330000000000004e-06 2024-09-13T04:58:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131278+0000", 2024-09-13T04:58:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9269999999999999e-06 2024-09-13T04:58:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131282+0000", 2024-09-13T04:58:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8729999999999998e-06 2024-09-13T04:58:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150497+0000", 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019214709 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150537+0000", 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0197000000000002e-05 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150591+0000", 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3352e-05 2024-09-13T04:58:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40001 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 638976~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.032086096, 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019422915999999998, 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40001 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131289+0000", 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.491e-06 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131306+0000", 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6489000000000001e-05 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131311+0000", 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150619+0000", 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019307642 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150657+0000", 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8015000000000001e-05 2024-09-13T04:58:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150710+0000", 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3109999999999998e-05 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40004 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 770048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.032057333999999, 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019514348000000001, 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40004 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131318+0000", 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8159999999999998e-06 2024-09-13T04:58:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131325+0000", 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5370000000000003e-06 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131330+0000", 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6619999999999996e-06 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150741+0000", 2024-09-13T04:58:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019410981000000001 2024-09-13T04:58:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150778+0000", 2024-09-13T04:58:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7793999999999998e-05 2024-09-13T04:58:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150830+0000", 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1974000000000001e-05 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40006 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 901120~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.032039419, 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019616892, 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40006 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131336+0000", 2024-09-13T04:58:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1940000000000001e-06 2024-09-13T04:58:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131342+0000", 2024-09-13T04:58:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0599999999999996e-06 2024-09-13T04:58:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131346+0000", 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5569999999999997e-06 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150858+0000", 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019512057999999999 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150897+0000", 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8903999999999999e-05 2024-09-13T04:58:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150951+0000", 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3467999999999997e-05 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40008 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1032192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.032022767000001, 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019721992000000001, 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40008 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131350+0000", 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131360+0000", 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5079999999999994e-06 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131367+0000", 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8210000000000004e-06 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131371+0000", 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8860000000000003e-06 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150979+0000", 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019607655000000002 2024-09-13T04:58:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151020+0000", 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0921999999999999e-05 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151073+0000", 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2567e-05 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40010 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1163264~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031997162, 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019817818000000001, 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40010 2024-09-13T04:58:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131378+0000", 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131384+0000", 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0780000000000001e-06 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131389+0000", 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5079999999999999e-06 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151101+0000", 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019712159 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151141+0000", 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0679000000000001e-05 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151194+0000", 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2589000000000001e-05 2024-09-13T04:58:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40012 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1294336~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031980805, 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019922019999999999, 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40012 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131392+0000", 2024-09-13T04:58:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131394+0000", 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.029e-06 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131400+0000", 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0560000000000001e-06 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131424+0000", 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3288000000000001e-05 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151223+0000", 2024-09-13T04:58:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019799231 2024-09-13T04:58:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151263+0000", 2024-09-13T04:58:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9920999999999999e-05 2024-09-13T04:58:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151315+0000", 2024-09-13T04:58:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1826000000000002e-05 2024-09-13T04:58:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40014 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1425408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031944269, 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020004087, 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40014 2024-09-13T04:58:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131432+0000", 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9780000000000001e-06 2024-09-13T04:58:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131441+0000", 2024-09-13T04:58:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9590000000000001e-06 2024-09-13T04:58:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131447+0000", 2024-09-13T04:58:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9410000000000004e-06 2024-09-13T04:58:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151343+0000", 2024-09-13T04:58:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019896113999999999 2024-09-13T04:58:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151381+0000", 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7969999999999997e-05 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151433+0000", 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2543000000000002e-05 2024-09-13T04:58:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40016 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1556480~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031891831999999, 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020070607000000001, 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40016 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131481+0000", 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131484+0000", 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.326e-06 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131492+0000", 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0139999999999998e-06 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131505+0000", 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151461+0000", 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01995535 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151500+0000", 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8760999999999997e-05 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151552+0000", 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2533e-05 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40018 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1687552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031862007999999, 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020160419999999998, 2024-09-13T04:58:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40018 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131514+0000", 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0740000000000001e-06 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131520+0000", 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4479999999999999e-06 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131526+0000", 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151580+0000", 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020054424000000001 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151619+0000", 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9229e-05 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151672+0000", 2024-09-13T04:58:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2493e-05 2024-09-13T04:58:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40019 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 114688~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031843035, 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020264445999999998, 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40019 2024-09-13T04:58:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131532+0000", 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.367e-06 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131538+0000", 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1070000000000001e-06 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131543+0000", 2024-09-13T04:58:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3930000000000001e-06 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151700+0000", 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020157191000000001 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151741+0000", 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0864e-05 2024-09-13T04:58:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151795+0000", 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3896e-05 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40020 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031826133999999, 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020368440000000002, 2024-09-13T04:58:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40020 2024-09-13T04:58:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131549+0000", 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1110000000000002e-06 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131555+0000", 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3350000000000003e-06 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131560+0000", 2024-09-13T04:58:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151823+0000", 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020262782999999999 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151863+0000", 2024-09-13T04:58:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0151000000000002e-05 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151916+0000", 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2604e-05 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40021 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1949696~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031809275000001, 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020470662000000001, 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40021 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131566+0000", 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.001e-06 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131571+0000", 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5110000000000003e-06 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131576+0000", 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151943+0000", 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020366649000000001 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151981+0000", 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8756e-05 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152035+0000", 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3335999999999999e-05 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40023 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031786842000001, 2024-09-13T04:58:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020567854, 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40023 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131589+0000", 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4140000000000001e-06 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131597+0000", 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5220000000000007e-06 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131613+0000", 2024-09-13T04:58:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5628999999999999e-05 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152063+0000", 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020450032 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152101+0000", 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7886999999999998e-05 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152154+0000", 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3739999999999997e-05 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40052 3.e 3:75db92e1:::rbd_header.112a6578f5c6:head [watch reconnect cookie 139644891391936 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.031754893, 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021419572000000001, 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40052 2024-09-13T04:58:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131621+0000", 2024-09-13T04:58:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9280000000000002e-06 2024-09-13T04:58:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131635+0000", 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4467000000000001e-05 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131642+0000", 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.144e-06 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152171+0000", 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020529544 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152220+0000", 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.153038+0000", 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00081775299999999995 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:49893 3.11 e48/24 3:88a8e927:::rbd_data.112ae3a54590.0000000000000005:head v 48'89, mlcod=47'87)", 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.1167840889999998, 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023644946999999999, 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048077+0000", 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001487817 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048104+0000", 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7056e-05 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048117+0000", 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2914999999999999e-05 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048200+0000", 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3042999999999998e-05 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048231+0000", 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0834999999999999e-05 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070186+0000", 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021955283999999999 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070234+0000", 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8325000000000002e-05 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.276 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:39.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:39.278 INFO:teuthology.orchestra.run.smithi001.stdout:2158 trunc from 0x60ef1dd to 0x4c3a1bb 2024-09-13T04:58:39.278 INFO:teuthology.orchestra.run.smithi001.stdout:2159 writesame 0x647c152 thru 0x6485382 data_size 0x5(0x9231 bytes) 2024-09-13T04:58:39.278 INFO:teuthology.orchestra.run.smithi001.stdout:2160 write 0x39e5e10 thru 0x39f4bba (0xedab bytes) 2024-09-13T04:58:39.278 INFO:teuthology.orchestra.run.smithi001.stdout:2161 punch from 0x3d596f3 to 0x3d68091, (0xe99e bytes) 2024-09-13T04:58:39.278 INFO:teuthology.orchestra.run.smithi001.stdout:2162 writesame 0xbb48875 thru 0xbb55009 data_size 0x15(0xc795 bytes) 2024-09-13T04:58:39.279 INFO:teuthology.orchestra.run.smithi001.stdout:2165 write 0xcc6e859 thru 0xcc7548e (0x6c36 bytes) 2024-09-13T04:58:39.279 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:39.275+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:39.280 INFO:teuthology.orchestra.run.smithi001.stdout:2166 punch from 0xa5c8683 to 0xa5d55a8, (0xcf25 bytes) 2024-09-13T04:58:39.281 INFO:teuthology.orchestra.run.smithi001.stdout:2169 writesame 0x9b17ce7 thru 0x9b2628f data_size 0x25(0xe5a9 bytes) 2024-09-13T04:58:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:2170 trunc from 0xcc7548f to 0x7ba9cce 2024-09-13T04:58:39.315 INFO:teuthology.orchestra.run.smithi001.stdout:2172 read 0x61508de thru 0x6151c8e (0x13b1 bytes) 2024-09-13T04:58:39.315 INFO:teuthology.orchestra.run.smithi001.stdout:2173 read 0x663af72 thru 0x66433d3 (0x8462 bytes) 2024-09-13T04:58:39.315 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.315 INFO:teuthology.orchestra.run.smithi001.stdout:2175 write 0x29c7c03 thru 0x29c8cd6 (0x10d4 bytes) 2024-09-13T04:58:39.341 INFO:teuthology.orchestra.run.smithi001.stdout:2177 write 0xd7ead35 thru 0xd7f45a0 (0x986c bytes) 2024-09-13T04:58:39.347 INFO:teuthology.orchestra.run.smithi001.stdout:2178 trunc from 0xd7f45a1 to 0xde8bc26 2024-09-13T04:58:39.354 INFO:teuthology.orchestra.run.smithi001.stdout:2179 punch from 0x3d13fa1 to 0x3d1cbe4, (0x8c43 bytes) 2024-09-13T04:58:39.354 INFO:teuthology.orchestra.run.smithi001.stdout:2180 write 0xc86dd80 thru 0xc872c2a (0x4eab bytes) 2024-09-13T04:58:39.361 INFO:teuthology.orchestra.run.smithi001.stdout:2181 punch from 0xa98e66d to 0xa993d17, (0x56aa bytes) 2024-09-13T04:58:39.361 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.361 INFO:teuthology.orchestra.run.smithi001.stdout:2183 punch from 0x79d0f26 to 0x79d438e, (0x3468 bytes) 2024-09-13T04:58:39.361 INFO:teuthology.orchestra.run.smithi001.stdout:2185 read 0xddfc011 thru 0xde0667a (0xa66a bytes) 2024-09-13T04:58:39.362 INFO:teuthology.orchestra.run.smithi001.stdout:2186 write 0x834f55c thru 0x835a772 (0xb217 bytes) 2024-09-13T04:58:39.371 INFO:teuthology.orchestra.run.smithi001.stdout:2187 write 0x643dc4e thru 0x64433b2 (0x5765 bytes) 2024-09-13T04:58:39.377 INFO:teuthology.orchestra.run.smithi001.stdout:2188 write 0x238383f thru 0x2389e6a (0x662c bytes) 2024-09-13T04:58:39.378 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:39.375+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:39.393 INFO:teuthology.orchestra.run.smithi001.stdout:2189 read 0x1724a48 thru 0x172e3d7 (0x9990 bytes) 2024-09-13T04:58:39.393 INFO:teuthology.orchestra.run.smithi001.stdout:2190 write 0x3310710 thru 0x33114a5 (0xd96 bytes) 2024-09-13T04:58:39.398 INFO:teuthology.orchestra.run.smithi001.stdout:2191 punch from 0x186ef03 to 0x1878209, (0x9306 bytes) 2024-09-13T04:58:39.398 INFO:teuthology.orchestra.run.smithi001.stdout:2192 read 0x7f792bf thru 0x7f817e7 (0x8529 bytes) 2024-09-13T04:58:39.407 INFO:teuthology.orchestra.run.smithi001.stdout:2195 write 0xe227e59 thru 0xe230951 (0x8af9 bytes) 2024-09-13T04:58:39.413 INFO:teuthology.orchestra.run.smithi001.stdout:2196 write 0xd9b7ab3 thru 0xd9c20ea (0xa638 bytes) 2024-09-13T04:58:39.421 INFO:teuthology.orchestra.run.smithi001.stdout:2197 read 0xcdb3d34 thru 0xcdb69dd (0x2caa bytes) 2024-09-13T04:58:39.421 INFO:teuthology.orchestra.run.smithi001.stdout:2198 write 0x3020035 thru 0x30297c8 (0x9794 bytes) 2024-09-13T04:58:39.430 INFO:teuthology.orchestra.run.smithi001.stdout:2199 write 0x54ad4a1 thru 0x54afe34 (0x2994 bytes) 2024-09-13T04:58:39.433 INFO:teuthology.orchestra.run.smithi001.stdout:2200 read 0x60f7c9a thru 0x61050b3 (0xd41a bytes) 2024-09-13T04:58:39.434 INFO:teuthology.orchestra.run.smithi001.stdout:2201 punch from 0xb8e2163 to 0xb8ef23b, (0xd0d8 bytes) 2024-09-13T04:58:39.434 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:39.434 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:39.434 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:39.434 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:39.434 INFO:teuthology.orchestra.run.smithi001.stdout:2202 write 0xd8a25d0 thru 0xd8ae13f (0xbb70 bytes) 2024-09-13T04:58:39.442 INFO:teuthology.orchestra.run.smithi001.stdout:2204 read 0x95b35f7 thru 0x95b57f1 (0x21fb bytes) 2024-09-13T04:58:39.442 INFO:teuthology.orchestra.run.smithi001.stdout:2206 read 0x84e8f72 thru 0x84f2acf (0x9b5e bytes) 2024-09-13T04:58:39.442 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.442 INFO:teuthology.orchestra.run.smithi001.stdout:2211 write 0x8ce0cdc thru 0x8cee39a (0xd6bf bytes) 2024-09-13T04:58:39.451 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:39.452 INFO:teuthology.orchestra.run.smithi001.stdout:2212 read 0xae9489b thru 0xae9a705 (0x5e6b bytes) 2024-09-13T04:58:39.452 INFO:teuthology.orchestra.run.smithi001.stdout:2213 read 0xa81e1c6 thru 0xa826383 (0x81be bytes) 2024-09-13T04:58:39.452 INFO:teuthology.orchestra.run.smithi001.stdout:2216 write 0xdcbcaed thru 0xdcbeb8f (0x20a3 bytes) 2024-09-13T04:58:39.456 INFO:teuthology.orchestra.run.smithi001.stdout:2217 write 0x1eaab84 thru 0x1eaf858 (0x4cd5 bytes) 2024-09-13T04:58:39.464 INFO:teuthology.orchestra.run.smithi001.stdout:2220 read 0x8b2b3ed thru 0x8b2f7da (0x43ee bytes) 2024-09-13T04:58:39.464 INFO:teuthology.orchestra.run.smithi001.stdout:2223 writesame 0xc0d5c70 thru 0xc0e48aa data_size 0x3b(0xec3b bytes) 2024-09-13T04:58:39.479 INFO:teuthology.orchestra.run.smithi001.stdout:2224 trunc from 0xe230952 to 0x9592348 2024-09-13T04:58:39.480 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:39.475+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:39.496 INFO:teuthology.orchestra.run.smithi001.stdout:2225 read 0x6c3b898 thru 0x6c3ff7d (0x46e6 bytes) 2024-09-13T04:58:39.496 INFO:teuthology.orchestra.run.smithi001.stdout:2226 read 0x7dba50c thru 0x7dc92b6 (0xedab bytes) 2024-09-13T04:58:39.496 INFO:teuthology.orchestra.run.smithi001.stdout:2227 punch from 0x78a35a8 to 0x78ad481, (0x9ed9 bytes) 2024-09-13T04:58:39.500 INFO:teuthology.orchestra.run.smithi001.stdout:2229 trunc from 0x9592348 to 0x6728ca3 2024-09-13T04:58:39.513 INFO:teuthology.orchestra.run.smithi001.stdout:2230 read 0x4f8625f thru 0x4f9170f (0xb4b1 bytes) 2024-09-13T04:58:39.525 INFO:teuthology.orchestra.run.smithi001.stdout:2231 write 0x9ebf901 thru 0x9ec30cf (0x37cf bytes) 2024-09-13T04:58:39.544 INFO:teuthology.orchestra.run.smithi001.stdout:2232 trunc from 0x9ec30d0 to 0x1794d3d 2024-09-13T04:58:39.580 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:39.575+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:39.595 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:39.595 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:39.596 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:39.596 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:39.596 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:39.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:39.620 INFO:teuthology.orchestra.run.smithi001.stdout:2235 read 0x6940ba thru 0x69c711 (0x8658 bytes) 2024-09-13T04:58:39.620 INFO:teuthology.orchestra.run.smithi001.stdout:2236 read 0x14e10c3 thru 0x14e6693 (0x55d1 bytes) 2024-09-13T04:58:39.620 INFO:teuthology.orchestra.run.smithi001.stdout:2237 read 0xad2a6c thru 0xadd1df (0xa774 bytes) 2024-09-13T04:58:39.622 INFO:teuthology.orchestra.run.smithi001.stdout:2238 punch from 0x3e3d4d to 0x3ede54, (0xa107 bytes) 2024-09-13T04:58:39.622 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.622 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.648 INFO:teuthology.orchestra.run.smithi001.stdout:2242 write 0xed01a7e thru 0xed06732 (0x4cb5 bytes) 2024-09-13T04:58:39.651 INFO:teuthology.orchestra.run.smithi001.stdout:2244 read 0xc868d3e thru 0xc868fea (0x2ad bytes) 2024-09-13T04:58:39.651 INFO:teuthology.orchestra.run.smithi001.stdout:2245 write 0xd9e0245 thru 0xd9e7fe5 (0x7da1 bytes) 2024-09-13T04:58:39.655 INFO:teuthology.orchestra.run.smithi001.stdout:2246 read 0x3e087d2 thru 0x3e159cb (0xd1fa bytes) 2024-09-13T04:58:39.655 INFO:teuthology.orchestra.run.smithi001.stdout:2249 punch from 0x88420eb to 0x884d8eb, (0xb800 bytes) 2024-09-13T04:58:39.655 INFO:teuthology.orchestra.run.smithi001.stdout:2251 write 0xcc20452 thru 0xcc2a294 (0x9e43 bytes) 2024-09-13T04:58:39.662 INFO:teuthology.orchestra.run.smithi001.stdout:2252 read 0xb30b069 thru 0xb312048 (0x6fe0 bytes) 2024-09-13T04:58:39.662 INFO:teuthology.orchestra.run.smithi001.stdout:2255 write 0xb4ef339 thru 0xb4f4666 (0x532e bytes) 2024-09-13T04:58:39.669 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.670 INFO:teuthology.orchestra.run.smithi001.stdout:2257 write 0x36fd039 thru 0x3709bf6 (0xcbbe bytes) 2024-09-13T04:58:39.678 INFO:teuthology.orchestra.run.smithi001.stdout:2258 writesame 0x2157f29 thru 0x215ac0d data_size 0x3(0x2ce5 bytes) 2024-09-13T04:58:39.681 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:39.679+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:39.685 INFO:teuthology.orchestra.run.smithi001.stdout:2260 write 0xe29e0cd thru 0xe2a4db7 (0x6ceb bytes) 2024-09-13T04:58:39.690 INFO:teuthology.orchestra.run.smithi001.stdout:2264 read 0x303c4c3 thru 0x3047471 (0xafaf bytes) 2024-09-13T04:58:39.691 INFO:teuthology.orchestra.run.smithi001.stdout:2265 write 0x899ea1d thru 0x899ecda (0x2be bytes) 2024-09-13T04:58:39.694 INFO:teuthology.orchestra.run.smithi001.stdout:2266 read 0x245bc06 thru 0x2462337 (0x6732 bytes) 2024-09-13T04:58:39.694 INFO:teuthology.orchestra.run.smithi001.stdout:2268 punch from 0x4d1921e to 0x4d1df5c, (0x4d3e bytes) 2024-09-13T04:58:39.694 INFO:teuthology.orchestra.run.smithi001.stdout:2269 write 0x51e1afa thru 0x51efc75 (0xe17c bytes) 2024-09-13T04:58:39.702 INFO:teuthology.orchestra.run.smithi001.stdout:2270 write 0x33c1da1 thru 0x33c4eb8 (0x3118 bytes) 2024-09-13T04:58:39.705 INFO:teuthology.orchestra.run.smithi001.stdout:2273 read 0xcf114c9 thru 0xcf19856 (0x838e bytes) 2024-09-13T04:58:39.705 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.705 INFO:teuthology.orchestra.run.smithi001.stdout:2276 read 0x9a9ecff thru 0x9aa25e8 (0x38ea bytes) 2024-09-13T04:58:39.706 INFO:teuthology.orchestra.run.smithi001.stdout:2277 writesame 0x6ce27d8 thru 0x6cf1a24 data_size 0x1(0xf24d bytes) 2024-09-13T04:58:39.748 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:58:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.583279736000001, 2024-09-13T04:58:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:58:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:58:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:58:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:58:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:58:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:58:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:58:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:58:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:58:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:58:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:58:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:58:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:58:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:58:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:58:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:58:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:58:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:15554 3.a 3:54a75321:::rbd_data.112af6c12117.0000000000000002:6 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.72191157, 2024-09-13T04:58:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047745517000000001, 2024-09-13T04:58:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 15554 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025346+0000", 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025348+0000", 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1900000000000002e-06 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025355+0000", 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7850000000000003e-06 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025360+0000", 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6789999999999996e-06 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.067347+0000", 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041987402 2024-09-13T04:58:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.072996+0000", 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056486540000000003 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.073092+0000", 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6151999999999997e-05 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.580839077, 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:58:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:58:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:58:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:58:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17859 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1458176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.456471197999999, 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047596271000000002, 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17859 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290789+0000", 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290794+0000", 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7610000000000004e-06 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290799+0000", 2024-09-13T04:58:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4889999999999997e-06 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333438+0000", 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042639200000000002 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333513+0000", 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5172000000000005e-05 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.338383+0000", 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048697929999999999 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17893 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452961784999999, 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046056773000000002, 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17893 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294298+0000", 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5000000000000002e-06 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294307+0000", 2024-09-13T04:58:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294312+0000", 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.451e-06 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340260+0000", 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045947904999999997 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340302+0000", 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1650999999999998e-05 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340353+0000", 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1208000000000002e-05 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17894 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452941279000001, 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046155512000000003, 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17894 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294316+0000", 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294319+0000", 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2390000000000001e-06 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294325+0000", 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0340000000000002e-06 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294329+0000", 2024-09-13T04:58:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7509999999999999e-06 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340378+0000", 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046048248 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340421+0000", 2024-09-13T04:58:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3627000000000001e-05 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340472+0000", 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17895 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452924495000001, 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046268562999999999, 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17895 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294333+0000", 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294335+0000", 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294341+0000", 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9939999999999997e-06 2024-09-13T04:58:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294346+0000", 2024-09-13T04:58:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7500000000000003e-06 2024-09-13T04:58:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340508+0000", 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046161354000000002 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340550+0000", 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2962e-05 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340602+0000", 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1665000000000001e-05 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17896 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1912832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452907747000001, 2024-09-13T04:58:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046367673999999998, 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17896 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294352+0000", 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.17e-06 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294358+0000", 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2249999999999997e-06 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294363+0000", 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6990000000000002e-06 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340627+0000", 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046263836000000003 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340668+0000", 2024-09-13T04:58:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0638e-05 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340718+0000", 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0458999999999999e-05 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17897 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1953792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452890937999999, 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046474117000000002, 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17897 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294369+0000", 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294375+0000", 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9880000000000001e-06 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294381+0000", 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7939999999999999e-06 2024-09-13T04:58:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340744+0000", 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046362962000000001 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340786+0000", 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2311000000000002e-05 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340841+0000", 2024-09-13T04:58:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5192999999999998e-05 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17898 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1994752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452872875000001, 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046575812000000001, 2024-09-13T04:58:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17898 2024-09-13T04:58:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294387+0000", 2024-09-13T04:58:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1600000000000001e-06 2024-09-13T04:58:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294395+0000", 2024-09-13T04:58:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.52e-06 2024-09-13T04:58:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294400+0000", 2024-09-13T04:58:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3149999999999999e-06 2024-09-13T04:58:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340868+0000", 2024-09-13T04:58:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046467893000000003 2024-09-13T04:58:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340911+0000", 2024-09-13T04:58:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3340000000000002e-05 2024-09-13T04:58:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340961+0000", 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9934e-05 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17899 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2035712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452854215999999, 2024-09-13T04:58:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691141999999998, 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17899 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294406+0000", 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294411+0000", 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294416+0000", 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5600000000000004e-06 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340987+0000", 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046570645000000001 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341045+0000", 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8065000000000001e-05 2024-09-13T04:58:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341095+0000", 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0302000000000001e-05 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17900 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2076672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452837753000001, 2024-09-13T04:58:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046796380999999998, 2024-09-13T04:58:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17900 2024-09-13T04:58:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294422+0000", 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9609999999999999e-06 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294428+0000", 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9290000000000003e-06 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294432+0000", 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4190000000000004e-06 2024-09-13T04:58:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341123+0000", 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046690597 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341165+0000", 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2289000000000001e-05 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341217+0000", 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1484999999999999e-05 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17901 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452821723, 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046918842000000002, 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17901 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294438+0000", 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9599999999999999e-06 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294444+0000", 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7740000000000001e-06 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294448+0000", 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5210000000000004e-06 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341242+0000", 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046794009999999997 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341288+0000", 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.596e-05 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341355+0000", 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6931e-05 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17904 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452805682000001, 2024-09-13T04:58:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006963999999998, 2024-09-13T04:58:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17904 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294454+0000", 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.979e-06 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294460+0000", 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7899999999999996e-06 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294464+0000", 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4139999999999996e-06 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341362+0000", 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046897498000000003 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341407+0000", 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4839000000000002e-05 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341459+0000", 2024-09-13T04:58:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2719000000000001e-05 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17911 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452789784, 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047111301000000001, 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17911 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294470+0000", 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9769999999999999e-06 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294476+0000", 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.925e-06 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294480+0000", 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5390000000000001e-06 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341487+0000", 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006829 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341530+0000", 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2503000000000003e-05 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341580+0000", 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9848000000000001e-05 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17914 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2048000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452773732000001, 2024-09-13T04:58:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047212684999999997, 2024-09-13T04:58:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17914 2024-09-13T04:58:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294486+0000", 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0899999999999999e-06 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294492+0000", 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1920000000000002e-06 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294497+0000", 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5399999999999997e-06 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341606+0000", 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047108783000000001 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341648+0000", 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1878999999999997e-05 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341697+0000", 2024-09-13T04:58:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9521999999999998e-05 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17916 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2088960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.452756924999999, 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047303235999999999, 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17916 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294506+0000", 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1880000000000001e-06 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294518+0000", 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1802e-05 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294523+0000", 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7219999999999999e-06 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341713+0000", 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190086999999999 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341755+0000", 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2530000000000001e-05 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341804+0000", 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9245e-05 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.291606782999999, 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:58:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:58:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:58:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:58:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:58:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:58:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.7340071360000007, 2024-09-13T04:58:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:58:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:58:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:58:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:58:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:58:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.7194960839999993, 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:58:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:58:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:39.871 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:39.872 INFO:teuthology.orchestra.run.smithi001.stdout:2278 punch from 0x53d6ea9 to 0x53de1fe, (0x7355 bytes) 2024-09-13T04:58:39.873 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.873 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.873 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.873 INFO:teuthology.orchestra.run.smithi001.stdout:2284 trunc from 0xed06733 to 0xd2e3bd8 2024-09-13T04:58:39.874 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:39.876 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2286 read 0xa9cce5a thru 0xa9d0c73 (0x3e1a bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2287 read 0xaf546ca thru 0xaf6037c (0xbcb3 bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2288 trunc from 0xd2e3bd8 to 0xc9eec12 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2290 punch from 0x8ef0b49 to 0x8ef755f, (0x6a16 bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2293 punch from 0x16f4980 to 0x16fdc93, (0x9313 bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2294 write 0xa479c90 thru 0xa487436 (0xd7a7 bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2296 read 0x5d0c10d thru 0x5d19dbf (0xdcb3 bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2297 punch from 0x34bfe2c to 0x34cbd5b, (0xbf2f bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2299 read 0xaaa912b thru 0xaaae234 (0x510a bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2300 punch from 0x261225a to 0x261fdf9, (0xdb9f bytes) 2024-09-13T04:58:39.877 INFO:teuthology.orchestra.run.smithi001.stdout:2301 read 0x4970f6b thru 0x497c6c8 (0xb75e bytes) 2024-09-13T04:58:39.878 INFO:teuthology.orchestra.run.smithi001.stdout:2303 read 0xc89541d thru 0xc89ba6f (0x6653 bytes) 2024-09-13T04:58:39.878 INFO:teuthology.orchestra.run.smithi001.stdout:2304 write 0x88b4951 thru 0x88ba487 (0x5b37 bytes) 2024-09-13T04:58:39.878 INFO:teuthology.orchestra.run.smithi001.stdout:2305 read 0x289492b thru 0x289f829 (0xaeff bytes) 2024-09-13T04:58:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:58:39.879 INFO:teuthology.orchestra.run.smithi001.stdout:2306 read 0x228fd03 thru 0x2299a36 (0x9d34 bytes) 2024-09-13T04:58:39.879 INFO:teuthology.orchestra.run.smithi001.stdout:2307 punch from 0x8ff963b to 0x8ffedaa, (0x576f bytes) 2024-09-13T04:58:39.879 INFO:teuthology.orchestra.run.smithi001.stdout:2308 punch from 0x76b5d3c to 0x76ba491, (0x4755 bytes) 2024-09-13T04:58:39.879 INFO:teuthology.orchestra.run.smithi001.stdout:2310 writesame 0x1fe1d3a thru 0x1fe53c9 data_size 0x48(0x3690 bytes) 2024-09-13T04:58:39.880 INFO:teuthology.orchestra.run.smithi001.stdout:2311 writesame 0x387d21b thru 0x387df51 data_size 0x1(0xd37 bytes) 2024-09-13T04:58:39.880 INFO:teuthology.orchestra.run.smithi001.stdout:2312 read 0x4204cfe thru 0x420792f (0x2c32 bytes) 2024-09-13T04:58:39.880 INFO:teuthology.orchestra.run.smithi001.stdout:2313 read 0x6567561 thru 0x6575642 (0xe0e2 bytes) 2024-09-13T04:58:39.880 INFO:teuthology.orchestra.run.smithi001.stdout:2314 writesame 0x32b8f2a thru 0x32c1d21 data_size 0x76(0x8df8 bytes) 2024-09-13T04:58:39.880 INFO:teuthology.orchestra.run.smithi001.stdout:2315 read 0xaf23afd thru 0xaf27f0f (0x4413 bytes) 2024-09-13T04:58:39.881 INFO:teuthology.orchestra.run.smithi001.stdout:2316 writesame 0xd94bacf thru 0xd94d5c3 data_size 0x1(0x1af5 bytes) 2024-09-13T04:58:39.881 INFO:teuthology.orchestra.run.smithi001.stdout:2317 read 0xa34d4e8 thru 0xa35ad47 (0xd860 bytes) 2024-09-13T04:58:39.881 INFO:teuthology.orchestra.run.smithi001.stdout:2318 writesame 0x4feeca6 thru 0x4ff7fb7 data_size 0x1(0x9312 bytes) 2024-09-13T04:58:39.882 INFO:teuthology.orchestra.run.smithi001.stdout:2319 write 0x4d007d7 thru 0x4d01a3a (0x1264 bytes) 2024-09-13T04:58:39.882 INFO:teuthology.orchestra.run.smithi001.stdout:2320 writesame 0x515e957 thru 0x516699e data_size 0x14(0x8048 bytes) 2024-09-13T04:58:39.882 INFO:teuthology.orchestra.run.smithi001.stdout:2321 write 0x4e12ac1 thru 0x4e1eb1f (0xc05f bytes) 2024-09-13T04:58:39.882 INFO:teuthology.orchestra.run.smithi001.stdout:2322 punch from 0x2616f8f to 0x261dbc1, (0x6c32 bytes) 2024-09-13T04:58:39.882 INFO:teuthology.orchestra.run.smithi001.stdout:2324 writesame 0x4f65e5b thru 0x4f6ffb0 data_size 0x2(0xa156 bytes) 2024-09-13T04:58:39.882 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:39.867+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:39.882 INFO:teuthology.orchestra.run.smithi001.stdout:2325 writesame 0x74c2d0f thru 0x74c9ad8 data_size 0x1a(0x6dca bytes) 2024-09-13T04:58:39.885 INFO:teuthology.orchestra.run.smithi001.stdout:2327 read 0xb95fd50 thru 0xb96b383 (0xb634 bytes) 2024-09-13T04:58:39.885 INFO:teuthology.orchestra.run.smithi001.stdout:2328 write 0xc347b70 thru 0xc34f42b (0x78bc bytes) 2024-09-13T04:58:39.891 INFO:teuthology.orchestra.run.smithi001.stdout:2329 punch from 0x48bced9 to 0x48c2e94, (0x5fbb bytes) 2024-09-13T04:58:39.891 INFO:teuthology.orchestra.run.smithi001.stdout:2332 punch from 0x3b22075 to 0x3b28667, (0x65f2 bytes) 2024-09-13T04:58:39.898 INFO:teuthology.orchestra.run.smithi001.stdout:2333 write 0xe1c4465 thru 0xe1cb8a3 (0x743f bytes) 2024-09-13T04:58:39.903 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.903 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.903 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:39.903 INFO:teuthology.orchestra.run.smithi001.stdout:2338 read 0x440e0d3 thru 0x44129a6 (0x48d4 bytes) 2024-09-13T04:58:39.903 INFO:teuthology.orchestra.run.smithi001.stdout:2339 read 0xcabf10b thru 0xcac511a (0x6010 bytes) 2024-09-13T04:58:39.903 INFO:teuthology.orchestra.run.smithi001.stdout:2340 write 0xa484ecd thru 0xa48edb6 (0x9eea bytes) 2024-09-13T04:58:39.910 INFO:teuthology.orchestra.run.smithi001.stdout:2341 read 0xdf777a thru 0xe0109c (0x9923 bytes) 2024-09-13T04:58:39.913 INFO:teuthology.orchestra.run.smithi001.stdout:2343 writesame 0xba47565 thru 0xba5215b data_size 0x13(0xabf7 bytes) 2024-09-13T04:58:39.919 INFO:teuthology.orchestra.run.smithi001.stdout:2344 write 0x5bdd480 thru 0x5be58fc (0x847d bytes) 2024-09-13T04:58:39.926 INFO:teuthology.orchestra.run.smithi001.stdout:2346 clone 15 order 18 su 4096 sc 13 2024-09-13T04:58:39.974 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:39.971+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:40.011 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:40.011 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:40.011 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:40.011 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:40.026 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:40.075 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:40.071+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:40.147 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:40.147 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:40.147 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:40.149 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:40.149 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:40.161 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:40.208 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:40.207+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:40.276 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:40.275+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:40.285 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:40.285 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:40.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15997 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1314816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256642702000001, 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088423697999999995, 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.286 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15997 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026558+0000", 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026564+0000", 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4399999999999996e-06 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026580+0000", 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6149000000000001e-05 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026593+0000", 2024-09-13T04:58:40.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2723e-05 2024-09-13T04:58:40.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114730+0000", 2024-09-13T04:58:40.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088137005000000004 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114857+0000", 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012730799999999999 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114983+0000", 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012595600000000001 2024-09-13T04:58:40.290 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15998 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256598174000001, 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088737302000000004, 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15998 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026603+0000", 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026608+0000", 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7440000000000004e-06 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026626+0000", 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8114999999999999e-05 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026638+0000", 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1976e-05 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115046+0000", 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088407311000000002 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115180+0000", 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013447700000000001 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115341+0000", 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000160676 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.295 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15999 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256551980000001, 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089223035000000006, 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.296 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15999 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026649+0000", 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026655+0000", 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9560000000000002e-06 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026671+0000", 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026684+0000", 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.273e-05 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115423+0000", 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088738368999999997 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115651+0000", 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022799699999999999 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115873+0000", 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222337 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16000 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256506022, 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089511479000000005, 2024-09-13T04:58:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16000 2024-09-13T04:58:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:40.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:40.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026695+0000", 2024-09-13T04:58:40.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026701+0000", 2024-09-13T04:58:40.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9000000000000003e-06 2024-09-13T04:58:40.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026717+0000", 2024-09-13T04:58:40.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6379e-05 2024-09-13T04:58:40.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026731+0000", 2024-09-13T04:58:40.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4039000000000001e-05 2024-09-13T04:58:40.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115965+0000", 2024-09-13T04:58:40.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089233554000000007 2024-09-13T04:58:40.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116078+0000", 2024-09-13T04:58:40.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000113655 2024-09-13T04:58:40.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116207+0000", 2024-09-13T04:58:40.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128942 2024-09-13T04:58:40.309 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.309 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.309 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.310 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16001 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1478656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.310 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:40.310 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256413057, 2024-09-13T04:58:40.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089757132000000003, 2024-09-13T04:58:40.310 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.311 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.311 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16001 2024-09-13T04:58:40.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.311 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026788+0000", 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026794+0000", 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8059999999999997e-06 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026827+0000", 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3002e-05 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026842+0000", 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4516e-05 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116278+0000", 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089435635999999999 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116414+0000", 2024-09-13T04:58:40.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000135919 2024-09-13T04:58:40.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116546+0000", 2024-09-13T04:58:40.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013235300000000001 2024-09-13T04:58:40.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.316 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.317 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16002 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1519616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.317 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:40.317 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256346090000001, 2024-09-13T04:58:40.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089990789000000002, 2024-09-13T04:58:40.317 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.375 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.375 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16002 2024-09-13T04:58:40.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.375 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026854+0000", 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026862+0000", 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5789999999999998e-06 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026883+0000", 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1892e-05 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026897+0000", 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.326e-05 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116609+0000", 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089712560999999996 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116708+0000", 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8787000000000001e-05 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116846+0000", 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013842999999999999 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16003 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256293462999999, 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090223747000000007, 2024-09-13T04:58:40.379 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16003 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026907+0000", 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026913+0000", 2024-09-13T04:58:40.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026930+0000", 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7065999999999999e-05 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026943+0000", 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3249e-05 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116903+0000", 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089959340999999998 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116992+0000", 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9690000000000004e-05 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117132+0000", 2024-09-13T04:58:40.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013984800000000001 2024-09-13T04:58:40.384 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.384 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.384 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.384 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16004 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1601536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.384 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256234259999999, 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090483628999999996, 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16004 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:40.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026967+0000", 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026978+0000", 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1219e-05 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027011+0000", 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3222000000000001e-05 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027025+0000", 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3995999999999999e-05 2024-09-13T04:58:40.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117203+0000", 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090178122999999999 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117287+0000", 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3721999999999996e-05 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117451+0000", 2024-09-13T04:58:40.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016434300000000001 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16005 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256163369999999, 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090781484999999995, 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.389 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16005 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027037+0000", 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027044+0000", 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4640000000000003e-06 2024-09-13T04:58:40.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027061+0000", 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6949e-05 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027074+0000", 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3422e-05 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117547+0000", 2024-09-13T04:58:40.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.09047318 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117682+0000", 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000134602 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117820+0000", 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013803200000000001 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.393 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16006 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256114269000001, 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091107120999999999, 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.394 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16006 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027086+0000", 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027092+0000", 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027109+0000", 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6330000000000001e-05 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027119+0000", 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0132999999999999e-05 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117876+0000", 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090756823 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118024+0000", 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000148758 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118195+0000", 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017021500000000001 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16007 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1724416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.398 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256075984999999, 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091267529999999999, 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16007 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:40.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027125+0000", 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027129+0000", 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5599999999999998e-06 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027137+0000", 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7129999999999994e-06 2024-09-13T04:58:40.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027144+0000", 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8329999999999996e-06 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118240+0000", 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091096166000000006 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118313+0000", 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000005e-05 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118393+0000", 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0680999999999995e-05 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16008 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1765376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.403 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.25605148, 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091580432000000003, 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16008 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:40.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027153+0000", 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3500000000000001e-06 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027161+0000", 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1799999999999996e-06 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027168+0000", 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5919999999999997e-06 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118454+0000", 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091286415999999995 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118592+0000", 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013743699999999999 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118731+0000", 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013906300000000001 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.407 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16009 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.256023587000001, 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091763040000000004, 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.408 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16009 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:40.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027177+0000", 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027181+0000", 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7160000000000001e-06 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027190+0000", 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4630000000000001e-06 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027196+0000", 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6429999999999998e-06 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118776+0000", 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091579259999999996 2024-09-13T04:58:40.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118855+0000", 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9695000000000002e-05 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118941+0000", 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6013000000000005e-05 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16010 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1847296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.255999721999999, 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091942769999999993, 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16010 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027201+0000", 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027205+0000", 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.213e-06 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027213+0000", 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2460000000000003e-06 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027220+0000", 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9970000000000001e-06 2024-09-13T04:58:40.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118986+0000", 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091766174000000006 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119056+0000", 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9988999999999997e-05 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119145+0000", 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8751000000000005e-05 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16011 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1888256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.255975747000001, 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130156000000005, 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16011 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:40.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027225+0000", 2024-09-13T04:58:40.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027229+0000", 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1640000000000001e-06 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027237+0000", 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3299999999999999e-06 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027243+0000", 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4620000000000001e-06 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119188+0000", 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091944247000000007 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119271+0000", 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3225000000000004e-05 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119356+0000", 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5325e-05 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16012 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.255952144999998, 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092295000000000002, 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16012 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027249+0000", 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027252+0000", 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0970000000000002e-06 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027260+0000", 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2959999999999999e-06 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027267+0000", 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6059999999999999e-06 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119398+0000", 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130904 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119466+0000", 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8460000000000005e-05 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119545+0000", 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8230000000000001e-05 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.426 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16013 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.255928886, 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092535714000000005, 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16013 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027272+0000", 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027275+0000", 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1130000000000001e-06 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027284+0000", 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2430000000000005e-06 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027290+0000", 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4999999999999996e-06 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119588+0000", 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092297660000000004 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119700+0000", 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000112526 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119809+0000", 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108262 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16014 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2011136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.255905823999999, 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092839985999999999, 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.431 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16014 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027295+0000", 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027298+0000", 2024-09-13T04:58:40.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1329999999999999e-06 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027307+0000", 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4859999999999997e-06 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027314+0000", 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5799999999999997e-06 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119880+0000", 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092566792999999994 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120027+0000", 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014627800000000001 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120136+0000", 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109329 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16015 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2052096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.255882409000002, 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093043843000000001, 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16015 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027322+0000", 2024-09-13T04:58:40.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.146e-06 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027330+0000", 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.524e-06 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027337+0000", 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4659999999999996e-06 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120194+0000", 2024-09-13T04:58:40.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092856578999999995 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120268+0000", 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4470999999999999e-05 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120363+0000", 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5253000000000004e-05 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.440 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.441 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16016 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:40.441 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:40.441 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.255858889999999, 2024-09-13T04:58:40.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093219623000000001, 2024-09-13T04:58:40.441 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16016 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027342+0000", 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027352+0000", 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0000000000000001e-05 2024-09-13T04:58:40.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027361+0000", 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7090000000000008e-06 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027368+0000", 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629999999999997e-06 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120393+0000", 2024-09-13T04:58:40.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093025262999999997 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120471+0000", 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7688000000000006e-05 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120563+0000", 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2003000000000007e-05 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:40.445 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:40.446 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:40.447 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:40.448 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:40.443+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:40.547 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:40.547+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:40.619 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:40.619 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:40.633 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:40.649 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:40.681 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:40.679+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:40.749 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:40.747+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:40.756 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:40.756 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:40.756 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:40.771 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:40.816 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:40.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:40.835 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:40.849 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:40.847+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:40.851 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:40.852 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:40.862 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone14 from 0xe1cb8a4 (overlap 0x1794d3d) to 0x7130c5 2024-09-13T04:58:40.873 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:40.873 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:40.896 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:40.896 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:40.918 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:40.942 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:40.950 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:40.947+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:40.953 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:40.953 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:40.971 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:40.972 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:40.972 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:41.057 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:41.055+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:41.126 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-13T04:58:41.140 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:41.140 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:41.152 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:41.147+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:41.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:41.254 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:41.251+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:41.325 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:41.325 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:41.339 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:41.355 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:41.351+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:41.456 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:41.451+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:41.462 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:41.462 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:41.476 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:41.557 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:41.555+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:41.600 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:41.600 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:41.614 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:41.658 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:41.655+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:41.668 INFO:teuthology.orchestra.run.smithi001.stdout:2349 read 0x379b0f5 thru 0x37a4d02 (0x9c0e bytes) 2024-09-13T04:58:41.669 INFO:teuthology.orchestra.run.smithi001.stdout:2351 write 0x79d524 thru 0x7a1717 (0x41f4 bytes) 2024-09-13T04:58:41.759 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:41.755+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:41.765 INFO:teuthology.orchestra.run.smithi001.stdout:2352 write 0x6a07fda thru 0x6a15958 (0xd97f bytes) 2024-09-13T04:58:41.787 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:41.787 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:41.804 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:41.805 INFO:teuthology.orchestra.run.smithi001.stdout:2353 read 0xd283a55 thru 0xd289dbe (0x636a bytes) 2024-09-13T04:58:41.809 INFO:teuthology.orchestra.run.smithi001.stdout:2354 trunc from 0xe1cb8a4 to 0xc48a5c8 2024-09-13T04:58:41.850 INFO:teuthology.orchestra.run.smithi001.stdout:2355 clone 16 order 18 su 131072 sc 3 2024-09-13T04:58:41.860 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:41.855+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:41.881 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:58:41.924 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:41.924 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:41.938 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:41.989 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:41.987+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:42.061 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:42.061 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:42.061 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:42.059+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:42.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:42.162 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:42.159+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:42.187 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:58:42.188 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:58:42.188 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T04:58:42.188 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T04:58:42.188 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T04:58:42.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T04:58:42.234 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:42.234 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:42.249 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:42.263 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:42.259+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:42.312 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:42.312 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T04:58:42.312 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:42.364 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:42.359+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:42.367 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:42.368 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:42.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:42.464 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:42.463+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:42.551 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:42.552 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:42.566 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:42.563+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:42.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:42.667 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:42.663+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:42.691 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:42.691 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:42.705 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:42.767 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:42.763+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:58:42.848 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:42.848 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:42.863 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:42.905 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:42.903+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:42.942 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone15 from 0xc48a5c8 (overlap 0xc48a5c8) to 0x62452e4 2024-09-13T04:58:42.969 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:42.967+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:43.001 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:43.067 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:43.067 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:43.069 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:43.067+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:43.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:43.170 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:43.167+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:43.245 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:43.270 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:43.267+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:43.276 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:43.277 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:43.304 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:43.304 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:43.329 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:43.329 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:43.329 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:43.329 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:43.329 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:43.335 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:43.335 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:43.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:43.356 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:43.356 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:43.371 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:43.367+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:43.374 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:43.374 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:43.393 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:43.393 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:43.414 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:43.465 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:43.465 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:43.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:43.465 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:43.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.465 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:58:43.465 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.003292871999999, 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.469 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:37637 3.11 e41/24 3:88b77d11:::rbd_data.112a66af081a.0000000000000000:head v 41'72, mlcod=41'70)", 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.109093915999999, 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027255564999999999, 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355356+0000", 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356532+0000", 2024-09-13T04:58:43.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001176599 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356563+0000", 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0916e-05 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356576+0000", 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3293e-05 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356700+0000", 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123779 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356746+0000", 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6242000000000002e-05 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382519+0000", 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025772856 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382614+0000", 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4315999999999996e-05 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39466 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.368146498, 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022036103000000001, 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39466 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096315+0000", 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096326+0000", 2024-09-13T04:58:43.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1596999999999999e-05 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096332+0000", 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8560000000000003e-06 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112144+0000", 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015812001999999999 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112220+0000", 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5354999999999999e-05 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.118342+0000", 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061219409999999997 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39999 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 376832~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:43.478 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.333217067, 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019234370000000001, 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39999 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:43.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131235+0000", 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131234+0000", 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131237+0000", 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0839999999999999e-06 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131253+0000", 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6920999999999999e-05 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131259+0000", 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.716e-06 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150375+0000", 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019115732999999999 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150416+0000", 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0791999999999998e-05 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150469+0000", 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3519e-05 2024-09-13T04:58:43.482 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40000 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 507904~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.333187772, 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019326534999999999, 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40000 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:43.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131271+0000", 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9330000000000004e-06 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131278+0000", 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9269999999999999e-06 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131282+0000", 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8729999999999998e-06 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150497+0000", 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019214709 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150537+0000", 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0197000000000002e-05 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150591+0000", 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3352e-05 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40001 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 638976~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:43.487 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.333164687, 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019422915999999998, 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40001 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131289+0000", 2024-09-13T04:58:43.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.491e-06 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131306+0000", 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6489000000000001e-05 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131311+0000", 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150619+0000", 2024-09-13T04:58:43.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019307642 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150657+0000", 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8015000000000001e-05 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150710+0000", 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3109999999999998e-05 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40004 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 770048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.333135925000001, 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019514348000000001, 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40004 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131318+0000", 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8159999999999998e-06 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131325+0000", 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5370000000000003e-06 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131330+0000", 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6619999999999996e-06 2024-09-13T04:58:43.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150741+0000", 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019410981000000001 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150778+0000", 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7793999999999998e-05 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150830+0000", 2024-09-13T04:58:43.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1974000000000001e-05 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40006 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 901120~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.33311801, 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019616892, 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40006 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131336+0000", 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1940000000000001e-06 2024-09-13T04:58:43.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131342+0000", 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0599999999999996e-06 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131346+0000", 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5569999999999997e-06 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150858+0000", 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019512057999999999 2024-09-13T04:58:43.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150897+0000", 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8903999999999999e-05 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150951+0000", 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3467999999999997e-05 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40008 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1032192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.333101358, 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019721992000000001, 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40008 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131350+0000", 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131360+0000", 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5079999999999994e-06 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131367+0000", 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8210000000000004e-06 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131371+0000", 2024-09-13T04:58:43.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8860000000000003e-06 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150979+0000", 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019607655000000002 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151020+0000", 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0921999999999999e-05 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151073+0000", 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2567e-05 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40010 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1163264~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.333075752999999, 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019817818000000001, 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40010 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131378+0000", 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131384+0000", 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0780000000000001e-06 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131389+0000", 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5079999999999999e-06 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151101+0000", 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019712159 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151141+0000", 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0679000000000001e-05 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151194+0000", 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2589000000000001e-05 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40012 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1294336~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:43.509 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.333059395999999, 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019922019999999999, 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40012 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131392+0000", 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131394+0000", 2024-09-13T04:58:43.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.029e-06 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131400+0000", 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0560000000000001e-06 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131424+0000", 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3288000000000001e-05 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151223+0000", 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019799231 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151263+0000", 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9920999999999999e-05 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151315+0000", 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1826000000000002e-05 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.513 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40014 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1425408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.33302286, 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020004087, 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40014 2024-09-13T04:58:43.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131432+0000", 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9780000000000001e-06 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131441+0000", 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9590000000000001e-06 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131447+0000", 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9410000000000004e-06 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151343+0000", 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019896113999999999 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151381+0000", 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7969999999999997e-05 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151433+0000", 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2543000000000002e-05 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40016 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1556480~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.332970423000001, 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020070607000000001, 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40016 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131481+0000", 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131484+0000", 2024-09-13T04:58:43.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.326e-06 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131492+0000", 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0139999999999998e-06 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131505+0000", 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151461+0000", 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01995535 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151500+0000", 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8760999999999997e-05 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151552+0000", 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2533e-05 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40018 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1687552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.332940599000001, 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020160419999999998, 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40018 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.523 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:43.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131514+0000", 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0740000000000001e-06 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131520+0000", 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4479999999999999e-06 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131526+0000", 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151580+0000", 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020054424000000001 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151619+0000", 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9229e-05 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151672+0000", 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2493e-05 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40019 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 114688~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.332921625999999, 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020264445999999998, 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40019 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131532+0000", 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.367e-06 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131538+0000", 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1070000000000001e-06 2024-09-13T04:58:43.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131543+0000", 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3930000000000001e-06 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151700+0000", 2024-09-13T04:58:43.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020157191000000001 2024-09-13T04:58:43.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151741+0000", 2024-09-13T04:58:43.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0864e-05 2024-09-13T04:58:43.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151795+0000", 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3896e-05 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40020 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.332904725000001, 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020368440000000002, 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40020 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131549+0000", 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1110000000000002e-06 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131555+0000", 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3350000000000003e-06 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131560+0000", 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151823+0000", 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020262782999999999 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151863+0000", 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0151000000000002e-05 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151916+0000", 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2604e-05 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40021 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1949696~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.537 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.332887866, 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020470662000000001, 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40021 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:43.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131566+0000", 2024-09-13T04:58:43.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.001e-06 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131571+0000", 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5110000000000003e-06 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131576+0000", 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151943+0000", 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020366649000000001 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151981+0000", 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8756e-05 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152035+0000", 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3335999999999999e-05 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.541 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40023 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.332865433, 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020567854, 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40023 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:43.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131589+0000", 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4140000000000001e-06 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131597+0000", 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5220000000000007e-06 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131613+0000", 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5628999999999999e-05 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152063+0000", 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020450032 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152101+0000", 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7886999999999998e-05 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152154+0000", 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3739999999999997e-05 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40052 3.e 3:75db92e1:::rbd_header.112a6578f5c6:head [watch reconnect cookie 139644891391936 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.332833484, 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021419572000000001, 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40052 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131621+0000", 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9280000000000002e-06 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131635+0000", 2024-09-13T04:58:43.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4467000000000001e-05 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131642+0000", 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.144e-06 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152171+0000", 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020529544 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152220+0000", 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.153038+0000", 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00081775299999999995 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:49893 3.11 e48/24 3:88a8e927:::rbd_data.112ae3a54590.0000000000000005:head v 48'89, mlcod=47'87)", 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:43.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.4178626799999998, 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023644946999999999, 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:43.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048077+0000", 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001487817 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048104+0000", 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7056e-05 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048117+0000", 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2914999999999999e-05 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048200+0000", 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3042999999999998e-05 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048231+0000", 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0834999999999999e-05 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070186+0000", 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021955283999999999 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070234+0000", 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8325000000000002e-05 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.554 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.555 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.555 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:43.556 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-13T04:58:43.556 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:43.557 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:43.551+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:43.608 INFO:teuthology.orchestra.run.smithi001.stdout:2356 write 0xed99102 thru 0xeda1ff7 (0x8ef6 bytes) 2024-09-13T04:58:43.631 INFO:teuthology.orchestra.run.smithi001.stdout:2358 trunc from 0xeda1ff8 to 0x597986a 2024-09-13T04:58:43.657 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:43.651+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:43.661 INFO:teuthology.orchestra.run.smithi001.stdout:2359 punch from 0x57395ae to 0x573f9d3, (0x6425 bytes) 2024-09-13T04:58:43.681 INFO:teuthology.orchestra.run.smithi001.stdout:2361 write 0x92f4c71 thru 0x92fc7e0 (0x7b70 bytes) 2024-09-13T04:58:43.688 INFO:teuthology.orchestra.run.smithi001.stdout:2363 read 0x62fd035 thru 0x6304efb (0x7ec7 bytes) 2024-09-13T04:58:43.688 INFO:teuthology.orchestra.run.smithi001.stdout:2364 writesame 0x4ce6ae7 thru 0x4ceaa8a data_size 0x2(0x3fa4 bytes) 2024-09-13T04:58:43.722 INFO:teuthology.orchestra.run.smithi001.stdout:2365 trunc from 0x92fc7e1 to 0x118275a 2024-09-13T04:58:43.742 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55830 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737530+0000", 2024-09-13T04:58:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0022103660000000001, 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022262330000000002, 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55830 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737530+0000", 2024-09-13T04:58:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737530+0000", 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737530+0000", 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737532+0000", 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737538+0000", 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1349999999999997e-06 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737543+0000", 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7929999999999997e-06 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55831 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737548+0000", 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0021928109999999998, 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002246896, 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55831 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737548+0000", 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737548+0000", 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737548+0000", 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737550+0000", 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5359999999999999e-06 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737556+0000", 2024-09-13T04:58:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2480000000000001e-06 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737561+0000", 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9230000000000001e-06 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55832 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737566+0000", 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0021749310000000002, 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022586749999999999, 2024-09-13T04:58:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55832 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737566+0000", 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737566+0000", 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737565+0000", 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737572+0000", 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4239999999999998e-06 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737579+0000", 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7569999999999998e-06 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737584+0000", 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1070000000000004e-06 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55833 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737589+0000", 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0021520699999999999, 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022917340000000001, 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55833 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737589+0000", 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737589+0000", 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737588+0000", 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737591+0000", 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.694e-06 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737597+0000", 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737602+0000", 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6909999999999996e-06 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55834 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737668+0000", 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0020729960000000001, 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022434619999999999, 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55834 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737668+0000", 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737668+0000", 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737666+0000", 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737678+0000", 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2473000000000001e-05 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737704+0000", 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5358999999999999e-05 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737723+0000", 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.891e-05 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55835 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737738+0000", 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0020025339999999998, 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022034049999999999, 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55835 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737738+0000", 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737738+0000", 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737737+0000", 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737744+0000", 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737764+0000", 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9406999999999999e-05 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737777+0000", 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3339000000000001e-05 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55836 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737790+0000", 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0019506390000000001, 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021806540000000002, 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55836 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737790+0000", 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737790+0000", 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737789+0000", 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737800+0000", 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1163e-05 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737819+0000", 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8609000000000001e-05 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737832+0000", 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2974e-05 2024-09-13T04:58:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55837 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738485+0000", 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0012553900000000001, 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001514163, 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55837 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738485+0000", 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738485+0000", 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738484+0000", 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738496+0000", 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2119999999999999e-05 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738519+0000", 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3396000000000001e-05 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738537+0000", 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7615999999999999e-05 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55838 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738551+0000", 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.001189472, 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001477119, 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55838 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738551+0000", 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738551+0000", 2024-09-13T04:58:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738550+0000", 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738558+0000", 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0029999999999994e-06 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738576+0000", 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8397e-05 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738591+0000", 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4124999999999999e-05 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55839 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738615+0000", 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0011254699999999999, 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001443946, 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55839 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738615+0000", 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738615+0000", 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738614+0000", 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738621+0000", 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2069999999999998e-06 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738641+0000", 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9386e-05 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738655+0000", 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4496e-05 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55840 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738706+0000", 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.001035022, 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001382042, 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55840 2024-09-13T04:58:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738706+0000", 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738706+0000", 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738704+0000", 2024-09-13T04:58:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738712+0000", 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6529999999999994e-06 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738731+0000", 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9380000000000001e-05 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738746+0000", 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4256e-05 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55841 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738761+0000", 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00097956999999999992, 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001357811, 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55841 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738761+0000", 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738761+0000", 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738760+0000", 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738767+0000", 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5470000000000002e-06 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738784+0000", 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6993000000000001e-05 2024-09-13T04:58:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738798+0000", 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3993e-05 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55842 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738812+0000", 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00092843099999999996, 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001351058, 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55842 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738812+0000", 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738812+0000", 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738811+0000", 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738818+0000", 2024-09-13T04:58:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3799999999999999e-06 2024-09-13T04:58:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738835+0000", 2024-09-13T04:58:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7425999999999999e-05 2024-09-13T04:58:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738853+0000", 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8142999999999999e-05 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55843 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738865+0000", 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00087553000000000001, 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013326010000000001, 2024-09-13T04:58:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55843 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738865+0000", 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738865+0000", 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738864+0000", 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738871+0000", 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4899999999999997e-06 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738888+0000", 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7416000000000001e-05 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738901+0000", 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3096e-05 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55844 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738913+0000", 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00082801800000000003, 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00131347, 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55844 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738913+0000", 2024-09-13T04:58:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738913+0000", 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738912+0000", 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738918+0000", 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.913e-06 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738935+0000", 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7102999999999999e-05 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738948+0000", 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3190000000000001e-05 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55845 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.738960+0000", 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00078089499999999998, 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012949439999999999, 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55845 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738960+0000", 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738960+0000", 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738959+0000", 2024-09-13T04:58:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.738965+0000", 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2899999999999999e-06 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739015+0000", 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9728000000000002e-05 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739031+0000", 2024-09-13T04:58:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6135999999999999e-05 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55846 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739045+0000", 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00069612700000000005, 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001238835, 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55846 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739045+0000", 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739045+0000", 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739044+0000", 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739051+0000", 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4699999999999996e-06 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739069+0000", 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7646e-05 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739081+0000", 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2680000000000001e-05 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55815 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737229+0000", 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0025122120000000002, 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030951569999999999, 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55815 2024-09-13T04:58:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737229+0000", 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737229+0000", 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737228+0000", 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737230+0000", 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1919999999999999e-06 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737237+0000", 2024-09-13T04:58:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.139e-06 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737241+0000", 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8369999999999996e-06 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.740162+0000", 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002920775 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.740318+0000", 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000156265 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55847 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739094+0000", 2024-09-13T04:58:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00064696600000000001, 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012716260000000001, 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55847 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739094+0000", 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739094+0000", 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739093+0000", 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739100+0000", 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7560000000000002e-06 2024-09-13T04:58:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739116+0000", 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6857000000000001e-05 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739129+0000", 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2976000000000001e-05 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55816 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737271+0000", 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024697349999999998, 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031728059999999998, 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55816 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737271+0000", 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737271+0000", 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737270+0000", 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737274+0000", 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9859999999999996e-06 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737283+0000", 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2010000000000005e-06 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737289+0000", 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9229999999999999e-06 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55848 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739164+0000", 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00057628099999999999, 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001318767, 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55848 2024-09-13T04:58:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739164+0000", 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739164+0000", 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739163+0000", 2024-09-13T04:58:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739170+0000", 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9999999999999999e-06 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739189+0000", 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9023000000000001e-05 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739204+0000", 2024-09-13T04:58:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4205999999999999e-05 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55817 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737296+0000", 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024450940000000001, 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032185199999999999, 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55817 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737296+0000", 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737296+0000", 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737295+0000", 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737298+0000", 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5119999999999998e-06 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737305+0000", 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8469999999999998e-06 2024-09-13T04:58:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737310+0000", 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1970000000000004e-06 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55849 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739216+0000", 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00052488600000000002, 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013267699999999999, 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55849 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739216+0000", 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739216+0000", 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739215+0000", 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739221+0000", 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2960000000000004e-06 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739239+0000", 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7887e-05 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739253+0000", 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3552e-05 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55818 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737315+0000", 2024-09-13T04:58:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.002426068, 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003258871, 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55818 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737315+0000", 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737315+0000", 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737314+0000", 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737317+0000", 2024-09-13T04:58:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.429e-06 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737323+0000", 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3969999999999999e-06 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737328+0000", 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5979999999999999e-06 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55850 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739264+0000", 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00047640999999999997, 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013376849999999999, 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55850 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739264+0000", 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739264+0000", 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739263+0000", 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739270+0000", 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1199999999999999e-06 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739287+0000", 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7499999999999998e-05 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739300+0000", 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2916e-05 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55819 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737332+0000", 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0024084060000000001, 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033005069999999998, 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55819 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737332+0000", 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737332+0000", 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737332+0000", 2024-09-13T04:58:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737334+0000", 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3549999999999999e-06 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737341+0000", 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1999999999999999e-06 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737345+0000", 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8899999999999998e-06 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55851 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739311+0000", 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00042928399999999999, 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001349977, 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55851 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739311+0000", 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739311+0000", 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739311+0000", 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739326+0000", 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5446000000000001e-05 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739344+0000", 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7986e-05 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739357+0000", 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2840000000000001e-05 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55820 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737354+0000", 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0023869759999999999, 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033370510000000002, 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55820 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737354+0000", 2024-09-13T04:58:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737354+0000", 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737353+0000", 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737356+0000", 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.571e-06 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737363+0000", 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4309999999999997e-06 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737368+0000", 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55852 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739369+0000", 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00037157599999999998, 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013498539999999999, 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55852 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739369+0000", 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739369+0000", 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739368+0000", 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739375+0000", 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4099999999999996e-06 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739391+0000", 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6589999999999999e-05 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739406+0000", 2024-09-13T04:58:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4746000000000001e-05 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55821 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737372+0000", 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0023683549999999999, 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034004489999999998, 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55821 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737372+0000", 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737372+0000", 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737372+0000", 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737374+0000", 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3590000000000002e-06 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737380+0000", 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0909999999999998e-06 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737385+0000", 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4240000000000003e-06 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55853 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739418+0000", 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00032292099999999999, 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001385215, 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55853 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739418+0000", 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739418+0000", 2024-09-13T04:58:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739417+0000", 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739423+0000", 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2740000000000004e-06 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739440+0000", 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6765999999999998e-05 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739453+0000", 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2696e-05 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55822 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737393+0000", 2024-09-13T04:58:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0023474699999999999, 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034438670000000002, 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55822 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737393+0000", 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737393+0000", 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737393+0000", 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737395+0000", 2024-09-13T04:58:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1500000000000002e-06 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737401+0000", 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9649999999999996e-06 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737406+0000", 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7759999999999997e-06 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55854 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739464+0000", 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00027657799999999998, 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014016829999999999, 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55854 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739464+0000", 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739464+0000", 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739463+0000", 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739469+0000", 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1330000000000004e-06 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739486+0000", 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6716000000000001e-05 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739499+0000", 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2717e-05 2024-09-13T04:58:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55823 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737410+0000", 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0023305819999999999, 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034884569999999999, 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55823 2024-09-13T04:58:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737410+0000", 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737410+0000", 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737410+0000", 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737412+0000", 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1720000000000001e-06 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737418+0000", 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.891e-06 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737422+0000", 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3259999999999997e-06 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55855 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739511+0000", 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00023005599999999999, 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014166140000000001, 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55855 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739511+0000", 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739511+0000", 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739510+0000", 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739516+0000", 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8900000000000004e-06 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739535+0000", 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8873e-05 2024-09-13T04:58:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739547+0000", 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2442999999999999e-05 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55824 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737426+0000", 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0023142900000000001, 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035635910000000001, 2024-09-13T04:58:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55824 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737426+0000", 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737426+0000", 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737426+0000", 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737428+0000", 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0829999999999998e-06 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737434+0000", 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737439+0000", 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4039999999999997e-06 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55856 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739560+0000", 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00018096099999999999, 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001465409, 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55856 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739560+0000", 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739560+0000", 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739559+0000", 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739565+0000", 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0460000000000002e-06 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739581+0000", 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6303e-05 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739594+0000", 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2653000000000001e-05 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55825 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737444+0000", 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.002297197, 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036445459999999998, 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55825 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737444+0000", 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737444+0000", 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737443+0000", 2024-09-13T04:58:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737446+0000", 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737452+0000", 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5030000000000002e-06 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737457+0000", 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4939999999999997e-06 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55857 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739606+0000", 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.00013425199999999999, 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015322199999999999, 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55857 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739606+0000", 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739606+0000", 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739605+0000", 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739612+0000", 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2929999999999997e-06 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739629+0000", 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7467000000000001e-05 2024-09-13T04:58:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739642+0000", 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2495e-05 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55826 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737460+0000", 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0022803260000000001, 2024-09-13T04:58:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037188310000000001, 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55826 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737460+0000", 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737460+0000", 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737460+0000", 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737462+0000", 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1660000000000001e-06 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737468+0000", 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0179999999999998e-06 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737473+0000", 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8210000000000001e-06 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55858 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739654+0000", 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.6979999999999997e-05, 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015682910000000001, 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55858 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739654+0000", 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739654+0000", 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739653+0000", 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739659+0000", 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0859999999999998e-06 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739676+0000", 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7085999999999999e-05 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739688+0000", 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2119999999999999e-05 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55827 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737478+0000", 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0022630160000000001, 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037833440000000001, 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55827 2024-09-13T04:58:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737478+0000", 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737478+0000", 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737477+0000", 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737480+0000", 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4549999999999998e-06 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737486+0000", 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.353e-06 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737491+0000", 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8210000000000001e-06 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55859 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739701+0000", 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9351000000000003e-05, 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001601147, 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55859 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739701+0000", 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739701+0000", 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739700+0000", 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739707+0000", 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.64e-06 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739726+0000", 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9315000000000001e-05 2024-09-13T04:58:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739739+0000", 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2794e-05 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55828 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737496+0000", 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.002245192, 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00384783, 2024-09-13T04:58:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55828 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737496+0000", 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737496+0000", 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737495+0000", 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737498+0000", 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5239999999999999e-06 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737504+0000", 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737509+0000", 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6480000000000002e-06 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55860 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739753+0000", 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9999876, 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001636591, 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55860 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739753+0000", 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739753+0000", 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739752+0000", 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739758+0000", 2024-09-13T04:58:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3470000000000004e-06 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739776+0000", 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7402999999999999e-05 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739792+0000", 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6252999999999998e-05 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55829 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.737513+0000", 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0022279270000000002, 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003907514, 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55829 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737513+0000", 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737513+0000", 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737512+0000", 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737515+0000", 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2989999999999999e-06 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737521+0000", 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.737526+0000", 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8389999999999997e-06 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:55861 3.d 3.e245268d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:43.739808+0000", 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4294967295.9999328, 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001644066, 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 55861 2024-09-13T04:58:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739808+0000", 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739808+0000", 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739807+0000", 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739812+0000", 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4839999999999998e-06 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739822+0000", 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7799999999999995e-06 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:43.739829+0000", 2024-09-13T04:58:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.046e-06 2024-09-13T04:58:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:58:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 47 2024-09-13T04:58:43.899 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:43.900 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:43.900 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:43.799+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:43.901 INFO:teuthology.orchestra.run.smithi001.stdout:2366 read 0xcbde4 thru 0xcc9da (0xbf7 bytes) 2024-09-13T04:58:43.901 INFO:teuthology.orchestra.run.smithi001.stdout:2367 writesame 0x71edbac thru 0x71ef086 data_size 0x1(0x14db bytes) 2024-09-13T04:58:43.901 INFO:teuthology.orchestra.run.smithi001.stdout:2369 punch from 0x128c70f to 0x129c0a7, (0xf998 bytes) 2024-09-13T04:58:43.901 INFO:teuthology.orchestra.run.smithi001.stdout:2370 punch from 0x22f690d to 0x2300f18, (0xa60b bytes) 2024-09-13T04:58:43.901 INFO:teuthology.orchestra.run.smithi001.stdout:2373 read 0x3854477 thru 0x38615ec (0xd176 bytes) 2024-09-13T04:58:43.902 INFO:teuthology.orchestra.run.smithi001.stdout:2375 trunc from 0x71ef087 to 0x2968d21 2024-09-13T04:58:43.902 INFO:teuthology.orchestra.run.smithi001.stdout:2376 read 0x5229a7 thru 0x52926f (0x68c9 bytes) 2024-09-13T04:58:43.902 INFO:teuthology.orchestra.run.smithi001.stdout:2378 writesame 0x681407b thru 0x6816bfd data_size 0x2f(0x2b83 bytes) 2024-09-13T04:58:43.902 INFO:teuthology.orchestra.run.smithi001.stdout:2379 read 0x3944ed9 thru 0x3953c5a (0xed82 bytes) 2024-09-13T04:58:43.902 INFO:teuthology.orchestra.run.smithi001.stdout:2380 trunc from 0x6816bfe to 0xe25bab5 2024-09-13T04:58:43.905 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:43.899+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:43.915 INFO:teuthology.orchestra.run.smithi001.stdout:2381 write 0x26d32e0 thru 0x26decb2 (0xb9d3 bytes) 2024-09-13T04:58:43.924 INFO:teuthology.orchestra.run.smithi001.stdout:2382 writesame 0xa50f046 thru 0xa51b1fe data_size 0x3(0xc1b9 bytes) 2024-09-13T04:58:43.941 INFO:teuthology.orchestra.run.smithi001.stdout:2383 writesame 0x1681143 thru 0x168e4df data_size 0x7(0xd39d bytes) 2024-09-13T04:58:43.956 INFO:teuthology.orchestra.run.smithi001.stdout:2385 writesame 0xc96486e thru 0xc9725f5 data_size 0x44(0xdd88 bytes) 2024-09-13T04:58:43.964 INFO:teuthology.orchestra.run.smithi001.stdout:2386 write 0x51b99f9 thru 0x51c7454 (0xda5c bytes) 2024-09-13T04:58:43.978 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:43.978 INFO:teuthology.orchestra.run.smithi001.stdout:2389 read 0x8073ce1 thru 0x80831f3 (0xf513 bytes) 2024-09-13T04:58:43.978 INFO:teuthology.orchestra.run.smithi001.stdout:2390 read 0xd285c10 thru 0xd293dfb (0xe1ec bytes) 2024-09-13T04:58:43.978 INFO:teuthology.orchestra.run.smithi001.stdout:2392 punch from 0x45bab9f to 0x45c756c, (0xc9cd bytes) 2024-09-13T04:58:43.978 INFO:teuthology.orchestra.run.smithi001.stdout:2395 read 0x1322700 thru 0x132698d (0x428e bytes) 2024-09-13T04:58:43.978 INFO:teuthology.orchestra.run.smithi001.stdout:2399 read 0x3807efc thru 0x3809dac (0x1eb1 bytes) 2024-09-13T04:58:43.979 INFO:teuthology.orchestra.run.smithi001.stdout:2400 read 0x4073a13 thru 0x408363d (0xfc2b bytes) 2024-09-13T04:58:43.979 INFO:teuthology.orchestra.run.smithi001.stdout:2402 write 0x85ff717 thru 0x8602bb8 (0x34a2 bytes) 2024-09-13T04:58:43.997 INFO:teuthology.orchestra.run.smithi001.stdout:2403 trunc from 0xe25bab5 to 0x7b0d7bd 2024-09-13T04:58:44.005 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:44.003+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:44.023 INFO:teuthology.orchestra.run.smithi001.stdout:2404 writesame 0x718b78d thru 0x7199083 data_size 0x1(0xd8f7 bytes) 2024-09-13T04:58:44.041 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:44.041 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:44.041 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:44.041 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:44.041 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:44.056 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:44.105 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:44.103+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:44.107 INFO:teuthology.orchestra.run.smithi001.stdout:2405 punch from 0x39369bd to 0x393f6d5, (0x8d18 bytes) 2024-09-13T04:58:44.108 INFO:teuthology.orchestra.run.smithi001.stdout:2406 read 0x2f61878 thru 0x2f66a67 (0x51f0 bytes) 2024-09-13T04:58:44.109 INFO:teuthology.orchestra.run.smithi001.stdout:2407 read 0x22e0013 thru 0x22eae14 (0xae02 bytes) 2024-09-13T04:58:44.109 INFO:teuthology.orchestra.run.smithi001.stdout:2409 write 0x3672d04 thru 0x3679877 (0x6b74 bytes) 2024-09-13T04:58:44.118 INFO:teuthology.orchestra.run.smithi001.stdout:2410 punch from 0x2958108 to 0x295ed41, (0x6c39 bytes) 2024-09-13T04:58:44.118 INFO:teuthology.orchestra.run.smithi001.stdout:2411 write 0x6f833d9 thru 0x6f83aef (0x717 bytes) 2024-09-13T04:58:44.125 INFO:teuthology.orchestra.run.smithi001.stdout:2412 punch from 0x2eb757d to 0x2ebb1a6, (0x3c29 bytes) 2024-09-13T04:58:44.125 INFO:teuthology.orchestra.run.smithi001.stdout:2413 read 0x5a9550c thru 0x5a9cc2e (0x7723 bytes) 2024-09-13T04:58:44.125 INFO:teuthology.orchestra.run.smithi001.stdout:2415 read 0xbe4ad8 thru 0xbee1ce (0x96f7 bytes) 2024-09-13T04:58:44.130 INFO:teuthology.orchestra.run.smithi001.stdout:2416 punch from 0x2fe06c9 to 0x2ff0563, (0xfe9a bytes) 2024-09-13T04:58:44.157 INFO:teuthology.orchestra.run.smithi001.stdout:2417 write 0xe6c69c4 thru 0xe6cf198 (0x87d5 bytes) 2024-09-13T04:58:44.179 INFO:teuthology.orchestra.run.smithi001.stdout:2418 trunc from 0xe6cf199 to 0x4ecad01 2024-09-13T04:58:44.206 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:44.203+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:44.232 INFO:teuthology.orchestra.run.smithi001.stdout:2420 write 0xcff5838 thru 0xcff9491 (0x3c5a bytes) 2024-09-13T04:58:44.238 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:44.238 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:44.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:44.238 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:44.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.238 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:58:44.238 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:44.238 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.071704551000003, 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:15554 3.a 3:54a75321:::rbd_data.112af6c12117.0000000000000002:6 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.210336385000002, 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047745517000000001, 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 15554 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:44.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025346+0000", 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025348+0000", 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1900000000000002e-06 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025355+0000", 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7850000000000003e-06 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025360+0000", 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6789999999999996e-06 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.067347+0000", 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041987402 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.072996+0000", 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056486540000000003 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.073092+0000", 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6151999999999997e-05 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:44.247 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.069263891999999, 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:58:44.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:44.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:58:44.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:58:44.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:58:44.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:58:44.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:58:44.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17859 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1458176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.944896013000001, 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047596271000000002, 2024-09-13T04:58:44.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17859 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290789+0000", 2024-09-13T04:58:44.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290794+0000", 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7610000000000004e-06 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290799+0000", 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4889999999999997e-06 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333438+0000", 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042639200000000002 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333513+0000", 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5172000000000005e-05 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.338383+0000", 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048697929999999999 2024-09-13T04:58:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17893 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941386600000001, 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046056773000000002, 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.258 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17893 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294298+0000", 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5000000000000002e-06 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294307+0000", 2024-09-13T04:58:44.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294312+0000", 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.451e-06 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340260+0000", 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045947904999999997 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340302+0000", 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1650999999999998e-05 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340353+0000", 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1208000000000002e-05 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.262 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17894 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941366093999999, 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046155512000000003, 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17894 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294316+0000", 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294319+0000", 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2390000000000001e-06 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294325+0000", 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0340000000000002e-06 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294329+0000", 2024-09-13T04:58:44.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7509999999999999e-06 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340378+0000", 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046048248 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340421+0000", 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3627000000000001e-05 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340472+0000", 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17895 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.94134931, 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046268562999999999, 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.267 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17895 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294333+0000", 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294335+0000", 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294341+0000", 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9939999999999997e-06 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294346+0000", 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7500000000000003e-06 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340508+0000", 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046161354000000002 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340550+0000", 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2962e-05 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340602+0000", 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1665000000000001e-05 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.271 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17896 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1912832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941332561999999, 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046367673999999998, 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17896 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294352+0000", 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.17e-06 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294358+0000", 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2249999999999997e-06 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294363+0000", 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6990000000000002e-06 2024-09-13T04:58:44.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340627+0000", 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046263836000000003 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340668+0000", 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0638e-05 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340718+0000", 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0458999999999999e-05 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17897 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1953792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941315753000001, 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046474117000000002, 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.276 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17897 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294369+0000", 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294375+0000", 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9880000000000001e-06 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294381+0000", 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7939999999999999e-06 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340744+0000", 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046362962000000001 2024-09-13T04:58:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340786+0000", 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2311000000000002e-05 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340841+0000", 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5192999999999998e-05 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17898 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1994752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941297689999999, 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046575812000000001, 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17898 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294387+0000", 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1600000000000001e-06 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294395+0000", 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.52e-06 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294400+0000", 2024-09-13T04:58:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3149999999999999e-06 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340868+0000", 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046467893000000003 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340911+0000", 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3340000000000002e-05 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340961+0000", 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9934e-05 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17899 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2035712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941279031000001, 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691141999999998, 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17899 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294406+0000", 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294411+0000", 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294416+0000", 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5600000000000004e-06 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340987+0000", 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046570645000000001 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341045+0000", 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8065000000000001e-05 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341095+0000", 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0302000000000001e-05 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17900 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2076672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941262567999999, 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046796380999999998, 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17900 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294422+0000", 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9609999999999999e-06 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294428+0000", 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9290000000000003e-06 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294432+0000", 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4190000000000004e-06 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341123+0000", 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046690597 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341165+0000", 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2289000000000001e-05 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341217+0000", 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1484999999999999e-05 2024-09-13T04:58:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17901 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941246538000001, 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046918842000000002, 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17901 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294438+0000", 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9599999999999999e-06 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294444+0000", 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7740000000000001e-06 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294448+0000", 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5210000000000004e-06 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341242+0000", 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046794009999999997 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341288+0000", 2024-09-13T04:58:44.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.596e-05 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341355+0000", 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6931e-05 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17904 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941230496999999, 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006963999999998, 2024-09-13T04:58:44.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17904 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294454+0000", 2024-09-13T04:58:44.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.979e-06 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294460+0000", 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7899999999999996e-06 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294464+0000", 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4139999999999996e-06 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341362+0000", 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046897498000000003 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341407+0000", 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4839000000000002e-05 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341459+0000", 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2719000000000001e-05 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17911 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941214598999998, 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047111301000000001, 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17911 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.303 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:44.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294470+0000", 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9769999999999999e-06 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294476+0000", 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.925e-06 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294480+0000", 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5390000000000001e-06 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341487+0000", 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006829 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341530+0000", 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2503000000000003e-05 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341580+0000", 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9848000000000001e-05 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17914 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2048000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941198546999999, 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047212684999999997, 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17914 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:44.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294486+0000", 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0899999999999999e-06 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294492+0000", 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1920000000000002e-06 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294497+0000", 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5399999999999997e-06 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341606+0000", 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047108783000000001 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341648+0000", 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1878999999999997e-05 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341697+0000", 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9521999999999998e-05 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17916 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2088960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.311 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.941181740000001, 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047303235999999999, 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17916 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294506+0000", 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1880000000000001e-06 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294518+0000", 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1802e-05 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294523+0000", 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7219999999999999e-06 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341713+0000", 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190086999999999 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341755+0000", 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2530000000000001e-05 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341804+0000", 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9245e-05 2024-09-13T04:58:44.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.780031598000001, 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:58:44.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:58:44.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.222431951000001, 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:58:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:58:44.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:58:44.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.207920898999999, 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.328 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:58:44.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:44.333 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:44.335 INFO:teuthology.orchestra.run.smithi001.stdout:2423 punch from 0x943ba37 to 0x943d9a9, (0x1f72 bytes) 2024-09-13T04:58:44.335 INFO:teuthology.orchestra.run.smithi001.stdout:2424 punch from 0x5767dc0 to 0x5771c81, (0x9ec1 bytes) 2024-09-13T04:58:44.335 INFO:teuthology.orchestra.run.smithi001.stdout:2425 writesame 0x6ddd04e thru 0x6ddd7c1 data_size 0x12(0x774 bytes) 2024-09-13T04:58:44.335 INFO:teuthology.orchestra.run.smithi001.stdout:2426 punch from 0x679ddbe to 0x67a36d5, (0x5917 bytes) 2024-09-13T04:58:44.335 INFO:teuthology.orchestra.run.smithi001.stdout:2427 write 0x8173e55 thru 0x817b3b0 (0x755c bytes) 2024-09-13T04:58:44.336 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:44.337 INFO:teuthology.orchestra.run.smithi001.stdout:2428 read 0xa25720f thru 0xa2636e7 (0xc4d9 bytes) 2024-09-13T04:58:44.337 INFO:teuthology.orchestra.run.smithi001.stdout:2429 writesame 0xd8a7e19 thru 0xd8aba57 data_size 0x35(0x3c3f bytes) 2024-09-13T04:58:44.337 INFO:teuthology.orchestra.run.smithi001.stdout:2431 writesame 0xd182957 thru 0xd18a87a data_size 0x4f(0x7f24 bytes) 2024-09-13T04:58:44.337 INFO:teuthology.orchestra.run.smithi001.stdout:2433 punch from 0x517a49b to 0x517c9d8, (0x253d bytes) 2024-09-13T04:58:44.338 INFO:teuthology.orchestra.run.smithi001.stdout:2434 punch from 0x5ce260d to 0x5cef7c4, (0xd1b7 bytes) 2024-09-13T04:58:44.338 INFO:teuthology.orchestra.run.smithi001.stdout:2435 write 0xe741360 thru 0xe749b92 (0x8833 bytes) 2024-09-13T04:58:44.338 INFO:teuthology.orchestra.run.smithi001.stdout:2437 write 0x3a62a2c thru 0x3a66a5e (0x4033 bytes) 2024-09-13T04:58:44.338 INFO:teuthology.orchestra.run.smithi001.stdout:2438 read 0x136f26d thru 0x136f963 (0x6f7 bytes) 2024-09-13T04:58:44.338 INFO:teuthology.orchestra.run.smithi001.stdout:2439 write 0x71c043 thru 0x723f25 (0x7ee3 bytes) 2024-09-13T04:58:44.338 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:44.331+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:44.343 INFO:teuthology.orchestra.run.smithi001.stdout:2440 read 0x67e0253 thru 0x67e922c (0x8fda bytes) 2024-09-13T04:58:44.343 INFO:teuthology.orchestra.run.smithi001.stdout:2442 read 0xa545e77 thru 0xa54f7b8 (0x9942 bytes) 2024-09-13T04:58:44.344 INFO:teuthology.orchestra.run.smithi001.stdout:2443 clone 17 order 23 su 65536 sc 3 2024-09-13T04:58:44.436 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:44.431+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:44.465 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:44.466 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:44.479 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:44.536 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:44.535+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:44.601 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:44.601 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:44.601 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:44.602 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:44.602 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:44.615 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:44.656 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:44.655+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:44.738 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:44.735+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:44.776 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15997 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1314816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747501349, 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088423697999999995, 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15997 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026558+0000", 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026564+0000", 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4399999999999996e-06 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026580+0000", 2024-09-13T04:58:44.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6149000000000001e-05 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026593+0000", 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2723e-05 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114730+0000", 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088137005000000004 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114857+0000", 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012730799999999999 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114983+0000", 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012595600000000001 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15998 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747456821, 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088737302000000004, 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15998 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026603+0000", 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026608+0000", 2024-09-13T04:58:44.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7440000000000004e-06 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026626+0000", 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8114999999999999e-05 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026638+0000", 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1976e-05 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115046+0000", 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088407311000000002 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115180+0000", 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013447700000000001 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115341+0000", 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000160676 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.785 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15999 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747410627000001, 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089223035000000006, 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15999 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.786 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026649+0000", 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026655+0000", 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9560000000000002e-06 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026671+0000", 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026684+0000", 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.273e-05 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115423+0000", 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088738368999999997 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115651+0000", 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022799699999999999 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115873+0000", 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222337 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16000 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747364669, 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089511479000000005, 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.790 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16000 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026695+0000", 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026701+0000", 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9000000000000003e-06 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026717+0000", 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6379e-05 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026731+0000", 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4039000000000001e-05 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115965+0000", 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089233554000000007 2024-09-13T04:58:44.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116078+0000", 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000113655 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116207+0000", 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128942 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16001 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1478656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747271703999999, 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089757132000000003, 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.795 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16001 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026788+0000", 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026794+0000", 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8059999999999997e-06 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026827+0000", 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3002e-05 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026842+0000", 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4516e-05 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116278+0000", 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089435635999999999 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116414+0000", 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000135919 2024-09-13T04:58:44.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116546+0000", 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013235300000000001 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16002 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1519616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747204737000001, 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089990789000000002, 2024-09-13T04:58:44.799 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16002 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026854+0000", 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026862+0000", 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5789999999999998e-06 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026883+0000", 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1892e-05 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026897+0000", 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.326e-05 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116609+0000", 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089712560999999996 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116708+0000", 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8787000000000001e-05 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116846+0000", 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013842999999999999 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.803 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16003 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747152110000002, 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090223747000000007, 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16003 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026907+0000", 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026913+0000", 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026930+0000", 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7065999999999999e-05 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026943+0000", 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3249e-05 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116903+0000", 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089959340999999998 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116992+0000", 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9690000000000004e-05 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117132+0000", 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013984800000000001 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16004 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1601536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747092906999999, 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090483628999999996, 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16004 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:44.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026967+0000", 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026978+0000", 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1219e-05 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027011+0000", 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3222000000000001e-05 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027025+0000", 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3995999999999999e-05 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117203+0000", 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090178122999999999 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117287+0000", 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3721999999999996e-05 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117451+0000", 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016434300000000001 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16005 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.812 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.747022016999999, 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090781484999999995, 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16005 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:44.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027037+0000", 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027044+0000", 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4640000000000003e-06 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027061+0000", 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6949e-05 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027074+0000", 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3422e-05 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117547+0000", 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.09047318 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117682+0000", 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000134602 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117820+0000", 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013803200000000001 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.816 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16006 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746972916000001, 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091107120999999999, 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16006 2024-09-13T04:58:44.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027086+0000", 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027092+0000", 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027109+0000", 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6330000000000001e-05 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027119+0000", 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0132999999999999e-05 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117876+0000", 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090756823 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118024+0000", 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000148758 2024-09-13T04:58:44.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118195+0000", 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017021500000000001 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16007 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1724416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746934631999999, 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091267529999999999, 2024-09-13T04:58:44.821 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16007 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027125+0000", 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027129+0000", 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5599999999999998e-06 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027137+0000", 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7129999999999994e-06 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027144+0000", 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8329999999999996e-06 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118240+0000", 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091096166000000006 2024-09-13T04:58:44.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118313+0000", 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000005e-05 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118393+0000", 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0680999999999995e-05 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16008 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1765376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746910127, 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091580432000000003, 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16008 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027153+0000", 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3500000000000001e-06 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027161+0000", 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1799999999999996e-06 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027168+0000", 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5919999999999997e-06 2024-09-13T04:58:44.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118454+0000", 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091286415999999995 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118592+0000", 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013743699999999999 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118731+0000", 2024-09-13T04:58:44.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013906300000000001 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16009 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746882234000001, 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091763040000000004, 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.830 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16009 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027177+0000", 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027181+0000", 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7160000000000001e-06 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027190+0000", 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4630000000000001e-06 2024-09-13T04:58:44.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027196+0000", 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6429999999999998e-06 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118776+0000", 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091579259999999996 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118855+0000", 2024-09-13T04:58:44.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9695000000000002e-05 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118941+0000", 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6013000000000005e-05 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16010 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1847296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746858369000002, 2024-09-13T04:58:44.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091942769999999993, 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16010 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027201+0000", 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027205+0000", 2024-09-13T04:58:44.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.213e-06 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027213+0000", 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2460000000000003e-06 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027220+0000", 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9970000000000001e-06 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118986+0000", 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091766174000000006 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119056+0000", 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9988999999999997e-05 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119145+0000", 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8751000000000005e-05 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16011 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1888256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746834394, 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130156000000005, 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16011 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.839 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027225+0000", 2024-09-13T04:58:44.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027229+0000", 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1640000000000001e-06 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027237+0000", 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3299999999999999e-06 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027243+0000", 2024-09-13T04:58:44.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4620000000000001e-06 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119188+0000", 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091944247000000007 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119271+0000", 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3225000000000004e-05 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119356+0000", 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5325e-05 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16012 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746810792000002, 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092295000000000002, 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.843 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16012 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027249+0000", 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027252+0000", 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0970000000000002e-06 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027260+0000", 2024-09-13T04:58:44.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2959999999999999e-06 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027267+0000", 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6059999999999999e-06 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119398+0000", 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130904 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119466+0000", 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8460000000000005e-05 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119545+0000", 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8230000000000001e-05 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16013 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:44.847 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746787532999999, 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092535714000000005, 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16013 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027272+0000", 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027275+0000", 2024-09-13T04:58:44.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1130000000000001e-06 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027284+0000", 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2430000000000005e-06 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027290+0000", 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4999999999999996e-06 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119588+0000", 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092297660000000004 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119700+0000", 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000112526 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119809+0000", 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108262 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16014 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2011136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746764470999999, 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092839985999999999, 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16014 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.852 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027295+0000", 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027298+0000", 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1329999999999999e-06 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027307+0000", 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4859999999999997e-06 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027314+0000", 2024-09-13T04:58:44.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5799999999999997e-06 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119880+0000", 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092566792999999994 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120027+0000", 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014627800000000001 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120136+0000", 2024-09-13T04:58:44.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109329 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16015 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2052096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746741056000001, 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093043843000000001, 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16015 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027322+0000", 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.146e-06 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027330+0000", 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.524e-06 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027337+0000", 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4659999999999996e-06 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120194+0000", 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092856578999999995 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120268+0000", 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4470999999999999e-05 2024-09-13T04:58:44.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120363+0000", 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5253000000000004e-05 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16016 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.746717536999999, 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093219623000000001, 2024-09-13T04:58:44.860 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16016 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027342+0000", 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027352+0000", 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0000000000000001e-05 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027361+0000", 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7090000000000008e-06 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027368+0000", 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629999999999997e-06 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120393+0000", 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093025262999999997 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120471+0000", 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7688000000000006e-05 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120563+0000", 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2003000000000007e-05 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.864 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.865 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.865 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:44.865 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:44.865 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:44.865 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:58:44.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:44.867 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:44.863+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:44.967 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:44.963+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:44.970 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone16 intact 2024-09-13T04:58:45.039 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:45.039 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:45.053 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:45.096 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:45.095+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:45.169 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:45.167+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:45.217 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-13T04:58:45.217 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:45.217 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:45.217 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:45.217 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:45.217 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:45.231 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:45.276 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:45.275+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:45.370 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:45.367+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:45.395 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:45.395 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:45.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:45.395 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:45.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.360307323000001, 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:58:45.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.360261686000001, 2024-09-13T04:58:45.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:58:45.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.360213602000002, 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:58:45.405 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:45.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:58:45.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:58:45.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.410 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.411 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.411 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:45.411 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.360117519999999, 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:58:45.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.360066149000001, 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.416 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.421 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.360020094999999, 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:58:45.425 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.359970034, 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.426 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:58:45.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:58:45.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.242554844000001, 2024-09-13T04:58:45.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:58:45.431 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.431 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:45.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.431 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:58:45.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.431 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:45.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:45.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:58:45.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.151126461, 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:45.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.442 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.151102116000001, 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:58:45.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.151020762999998, 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:58:45.447 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:58:45.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:58:45.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150951233000001, 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:58:45.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:58:45.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150923957, 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:58:45.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.460 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150825512000001, 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:58:45.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:58:45.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150755945, 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:58:45.465 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:58:45.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150685470999999, 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150659785999999, 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.478 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150624598, 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:58:45.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:58:45.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150600082, 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:58:45.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:58:45.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.150530745000001, 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:58:45.487 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:58:45.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:58:45.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.491 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:45.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:45.492 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:45.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:45.494 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:45.494 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:45.491+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:45.595 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:45.591+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:45.665 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:45.666 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:45.672 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:45.672 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:45.672 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:45.672 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:45.688 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:45.689 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:45.689 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:45.696 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:45.691+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:45.713 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:45.714 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:45.738 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:45.738 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:45.757 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:45.757 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:45.776 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:45.777 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:45.796 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:45.797 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:45.797 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:45.795+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:45.820 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:45.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:45.842 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:45.842 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:58:45.860 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:45.933 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:45.931+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:45.999 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:45.995+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:46.032 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:46.033 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:46.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:46.100 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:46.095+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:46.130 INFO:teuthology.orchestra.run.smithi001.stdout:2444 write 0x93f742b thru 0x93fb022 (0x3bf8 bytes) 2024-09-13T04:58:46.143 INFO:teuthology.orchestra.run.smithi001.stdout:2445 writesame 0xcc2c777 thru 0xcc340d4 data_size 0x41(0x795e bytes) 2024-09-13T04:58:46.152 INFO:teuthology.orchestra.run.smithi001.stdout:2449 write 0xc4f8358 thru 0xc4fd078 (0x4d21 bytes) 2024-09-13T04:58:46.161 INFO:teuthology.orchestra.run.smithi001.stdout:2450 write 0x6391408 thru 0x6391dd0 (0x9c9 bytes) 2024-09-13T04:58:46.167 INFO:teuthology.orchestra.run.smithi001.stdout:2453 punch from 0x59abdb8 to 0x59b3ab3, (0x7cfb bytes) 2024-09-13T04:58:46.167 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:46.167 INFO:teuthology.orchestra.run.smithi001.stdout:2456 punch from 0xcea254c to 0xceac4dc, (0x9f90 bytes) 2024-09-13T04:58:46.167 INFO:teuthology.orchestra.run.smithi001.stdout:2457 read 0xa0bc61 thru 0xa1994f (0xdcef bytes) 2024-09-13T04:58:46.171 INFO:teuthology.orchestra.run.smithi001.stdout:2459 read 0x8e65994 thru 0x8e747a5 (0xee12 bytes) 2024-09-13T04:58:46.171 INFO:teuthology.orchestra.run.smithi001.stdout:2461 read 0xa0e9904 thru 0xa0ede6d (0x456a bytes) 2024-09-13T04:58:46.171 INFO:teuthology.orchestra.run.smithi001.stdout:2465 read 0xd454e5b thru 0xd45743c (0x25e2 bytes) 2024-09-13T04:58:46.172 INFO:teuthology.orchestra.run.smithi001.stdout:2467 read 0x346d7cb thru 0x346e144 (0x97a bytes) 2024-09-13T04:58:46.172 INFO:teuthology.orchestra.run.smithi001.stdout:2470 read 0x1114eca thru 0x112053b (0xb672 bytes) 2024-09-13T04:58:46.176 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:46.176 INFO:teuthology.orchestra.run.smithi001.stdout:2472 write 0x460b273 thru 0x461442e (0x91bc bytes) 2024-09-13T04:58:46.188 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:46.188 INFO:teuthology.orchestra.run.smithi001.stdout:2475 clone 18 order 20 su 131072 sc 15 2024-09-13T04:58:46.201 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:46.199+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:46.210 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:46.210 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:46.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:46.302 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:46.299+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:46.390 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:46.391 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:46.402 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:46.399+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:46.404 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:46.503 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:46.503+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:46.526 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:46.526 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:46.540 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:46.606 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:46.603+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:46.675 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:46.675 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:46.690 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:46.706 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:46.703+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:46.807 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:46.803+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:46.813 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:46.813 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:46.827 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:46.908 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:46.903+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:46.962 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:46.962 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:46.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:47.023 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone17 from 0xe749b93 (overlap 0xe749b93) to 0xb907c75 2024-09-13T04:58:47.024 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:47.019+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:47.110 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:47.107+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:47.161 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:47.162 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:47.180 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:47.225 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:47.223+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:47.308 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-13T04:58:47.311 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:47.307+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:47.328 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:58:47.328 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:58:47.328 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T04:58:47.328 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T04:58:47.328 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T04:58:47.328 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T04:58:47.356 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:47.356 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:47.375 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:47.424 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:47.419+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:47.482 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:47.482 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T04:58:47.482 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:47.512 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:47.511+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:47.528 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:47.528 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:47.544 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:47.613 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:47.611+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:47.714 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:47.711+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:47.717 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:47.717 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:47.732 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:47.816 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:47.811+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:47.852 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:47.856 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:47.856 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:47.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:47.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:58:47.916 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:47.915+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:48.017 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:48.015+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:48.040 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:48.040 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:48.052 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:48.052 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:48.060 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:48.061 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:48.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:48.088 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:48.088 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:48.118 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:48.119 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:48.115+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:48.147 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:48.147 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:48.170 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:48.170 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:48.187 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:48.188 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:48.206 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:48.207 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:48.220 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:48.215+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:48.228 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:48.228 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:48.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:48.321 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:48.319+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:48.404 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:48.405 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:48.418 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:48.421 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:48.419+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:48.522 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:48.519+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:48.554 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.387339392999998, 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:58:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:15554 3.a 3:54a75321:::rbd_data.112af6c12117.0000000000000002:6 [read 1011712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.525971226999999, 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047745517000000001, 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 15554 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025347+0000", 2024-09-13T04:58:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025346+0000", 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025348+0000", 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1900000000000002e-06 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025355+0000", 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7850000000000003e-06 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.025360+0000", 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6789999999999996e-06 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.067347+0000", 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041987402 2024-09-13T04:58:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.072996+0000", 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056486540000000003 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.073092+0000", 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6151999999999997e-05 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.384898733999997, 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:58:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:58:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17859 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1458176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.260530854999999, 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047596271000000002, 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17859 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290787+0000", 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290789+0000", 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1579999999999999e-06 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290794+0000", 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7610000000000004e-06 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.290799+0000", 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4889999999999997e-06 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333438+0000", 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042639200000000002 2024-09-13T04:58:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.333513+0000", 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5172000000000005e-05 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.338383+0000", 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048697929999999999 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17893 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.257021442000003, 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046056773000000002, 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17893 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294296+0000", 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294298+0000", 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5000000000000002e-06 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294307+0000", 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294312+0000", 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.451e-06 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340260+0000", 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045947904999999997 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340302+0000", 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1650999999999998e-05 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340353+0000", 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1208000000000002e-05 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17894 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.257000935999997, 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046155512000000003, 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17894 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294317+0000", 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294316+0000", 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294319+0000", 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2390000000000001e-06 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294325+0000", 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0340000000000002e-06 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294329+0000", 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7509999999999999e-06 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340378+0000", 2024-09-13T04:58:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046048248 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340421+0000", 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3627000000000001e-05 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340472+0000", 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0992000000000002e-05 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17895 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256984152000001, 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046268562999999999, 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17895 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294334+0000", 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294333+0000", 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294335+0000", 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1890000000000001e-06 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294341+0000", 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9939999999999997e-06 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294346+0000", 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7500000000000003e-06 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340508+0000", 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046161354000000002 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340550+0000", 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2962e-05 2024-09-13T04:58:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340602+0000", 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1665000000000001e-05 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17896 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1912832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256967404000001, 2024-09-13T04:58:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046367673999999998, 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17896 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294350+0000", 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294352+0000", 2024-09-13T04:58:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.17e-06 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294358+0000", 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2249999999999997e-06 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294363+0000", 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6990000000000002e-06 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340627+0000", 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046263836000000003 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340668+0000", 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0638e-05 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340718+0000", 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0458999999999999e-05 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17897 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1953792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256950594999999, 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046474117000000002, 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17897 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294367+0000", 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294369+0000", 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294375+0000", 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9880000000000001e-06 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294381+0000", 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7939999999999999e-06 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340744+0000", 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046362962000000001 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340786+0000", 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2311000000000002e-05 2024-09-13T04:58:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340841+0000", 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5192999999999998e-05 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17898 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1994752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256932532, 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046575812000000001, 2024-09-13T04:58:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17898 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294385+0000", 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294387+0000", 2024-09-13T04:58:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1600000000000001e-06 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294395+0000", 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.52e-06 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294400+0000", 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3149999999999999e-06 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340868+0000", 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046467893000000003 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340911+0000", 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3340000000000002e-05 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340961+0000", 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9934e-05 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17899 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2035712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256913873000002, 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046691141999999998, 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17899 2024-09-13T04:58:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294404+0000", 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294406+0000", 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.08e-06 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294411+0000", 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294416+0000", 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5600000000000004e-06 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.340987+0000", 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046570645000000001 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341045+0000", 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8065000000000001e-05 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341095+0000", 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0302000000000001e-05 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17900 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2076672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256897410000001, 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046796380999999998, 2024-09-13T04:58:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17900 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294420+0000", 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294422+0000", 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9609999999999999e-06 2024-09-13T04:58:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294428+0000", 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9290000000000003e-06 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294432+0000", 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4190000000000004e-06 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341123+0000", 2024-09-13T04:58:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046690597 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341165+0000", 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2289000000000001e-05 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341217+0000", 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1484999999999999e-05 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17901 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1925120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256881380000003, 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046918842000000002, 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17901 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294436+0000", 2024-09-13T04:58:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294438+0000", 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9599999999999999e-06 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294444+0000", 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7740000000000001e-06 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294448+0000", 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5210000000000004e-06 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341242+0000", 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046794009999999997 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341288+0000", 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.596e-05 2024-09-13T04:58:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341355+0000", 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6931e-05 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17904 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256865339000001, 2024-09-13T04:58:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006963999999998, 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17904 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294452+0000", 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294454+0000", 2024-09-13T04:58:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.979e-06 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294460+0000", 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7899999999999996e-06 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294464+0000", 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4139999999999996e-06 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341362+0000", 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046897498000000003 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341407+0000", 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4839000000000002e-05 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341459+0000", 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2719000000000001e-05 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17911 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256849441, 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047111301000000001, 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17911 2024-09-13T04:58:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294468+0000", 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294470+0000", 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9769999999999999e-06 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294476+0000", 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.925e-06 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294480+0000", 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5390000000000001e-06 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341487+0000", 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047006829 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341530+0000", 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2503000000000003e-05 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341580+0000", 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9848000000000001e-05 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17914 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2048000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256833389000001, 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047212684999999997, 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17914 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294484+0000", 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294486+0000", 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0899999999999999e-06 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294492+0000", 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1920000000000002e-06 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294497+0000", 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5399999999999997e-06 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341606+0000", 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047108783000000001 2024-09-13T04:58:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341648+0000", 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1878999999999997e-05 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341697+0000", 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9521999999999998e-05 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:17916 3.14 3:29474653:::rbd_data.112af6c12117.0000000000000000:6 [read 2088960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.256816581999999, 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047303235999999999, 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 17916 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294501+0000", 2024-09-13T04:58:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294506+0000", 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1880000000000001e-06 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294518+0000", 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1802e-05 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.294523+0000", 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7219999999999999e-06 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341713+0000", 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047190086999999999 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341755+0000", 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2530000000000001e-05 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.341804+0000", 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9245e-05 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.095666440000002, 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:58:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:58:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:58:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.538066792999999, 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:58:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.523555740999999, 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:58:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:58:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:58:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:48.650 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:48.651 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:48.652 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:48.647+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:48.753 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:48.751+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:48.782 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:48.782 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:48.782 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:48.782 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:48.796 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:48.817 INFO:teuthology.orchestra.run.smithi001.stdout:2476 read 0xa160ece thru 0xa167378 (0x64ab bytes) 2024-09-13T04:58:48.826 INFO:teuthology.orchestra.run.smithi001.stdout:2478 trunc from 0xe749b93 to 0x905ce69 2024-09-13T04:58:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:58:48.873 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:48.879 INFO:teuthology.orchestra.run.smithi001.stdout:2481 trunc from 0x905ce69 to 0xd29d96a 2024-09-13T04:58:48.884 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:48.883+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:48.887 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:48.887 INFO:teuthology.orchestra.run.smithi001.stdout:2483 write 0xb978b03 thru 0xb980084 (0x7582 bytes) 2024-09-13T04:58:48.893 INFO:teuthology.orchestra.run.smithi001.stdout:2485 write 0x7a41fe8 thru 0x7a4c352 (0xa36b bytes) 2024-09-13T04:58:48.914 INFO:teuthology.orchestra.run.smithi001.stdout:2486 trunc from 0xd29d96a to 0x2f03b4d 2024-09-13T04:58:48.953 INFO:teuthology.orchestra.run.smithi001.stdout:2487 punch from 0x16e3f83 to 0x16f0539, (0xc5b6 bytes) 2024-09-13T04:58:48.953 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:48.951+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:48.962 INFO:teuthology.orchestra.run.smithi001.stdout:2489 writesame 0x4bc3e1f thru 0x4bc5f70 data_size 0x5(0x2152 bytes) 2024-09-13T04:58:48.975 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:48.976 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:48.983 INFO:teuthology.orchestra.run.smithi001.stdout:2490 write 0xe83f5c1 thru 0xe84ce89 (0xd8c9 bytes) 2024-09-13T04:58:48.987 INFO:teuthology.orchestra.run.smithi001.stdout:2491 write 0xadb2b5b thru 0xadb38db (0xd81 bytes) 2024-09-13T04:58:48.990 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:49.007 INFO:teuthology.orchestra.run.smithi001.stdout:2492 trunc from 0xe84ce8a to 0x3a15a42 2024-09-13T04:58:49.025 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.027 INFO:teuthology.orchestra.run.smithi001.stdout:2494 trunc from 0x3a15a42 to 0x56ed3be 2024-09-13T04:58:49.036 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.036 INFO:teuthology.orchestra.run.smithi001.stdout:2496 punch from 0x4ea1e0d to 0x4eab5c6, (0x97b9 bytes) 2024-09-13T04:58:49.037 INFO:teuthology.orchestra.run.smithi001.stdout:2498 write 0x270296d thru 0x270e9af (0xc043 bytes) 2024-09-13T04:58:49.046 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.065 INFO:teuthology.orchestra.run.smithi001.stdout:2500 write 0xbec49a1 thru 0xbec5963 (0xfc3 bytes) 2024-09-13T04:58:49.067 INFO:teuthology.orchestra.run.smithi001.stdout:2501 read 0x49f6e50 thru 0x49f7e52 (0x1003 bytes) 2024-09-13T04:58:49.068 INFO:teuthology.orchestra.run.smithi001.stdout:2503 trunc from 0xbec5964 to 0xc27cbcc 2024-09-13T04:58:49.076 INFO:teuthology.orchestra.run.smithi001.stdout:2504 read 0x1c6b2e1 thru 0x1c78528 (0xd248 bytes) 2024-09-13T04:58:49.080 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:49.079+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:49.088 INFO:teuthology.orchestra.run.smithi001.stdout:2505 write 0xd75a15e thru 0xd7697da (0xf67d bytes) 2024-09-13T04:58:49.094 INFO:teuthology.orchestra.run.smithi001.stdout:2506 write 0x789b82f thru 0x78a3ed6 (0x86a8 bytes) 2024-09-13T04:58:49.102 INFO:teuthology.orchestra.run.smithi001.stdout:2510 write 0xa7f5cfe thru 0xa801ee1 (0xc1e4 bytes) 2024-09-13T04:58:49.110 INFO:teuthology.orchestra.run.smithi001.stdout:2511 read 0xae8c0b1 thru 0xae8f44d (0x339d bytes) 2024-09-13T04:58:49.110 INFO:teuthology.orchestra.run.smithi001.stdout:2512 read 0x44034cc thru 0x440f2ad (0xbde2 bytes) 2024-09-13T04:58:49.111 INFO:teuthology.orchestra.run.smithi001.stdout:2515 write 0xb8ce04f thru 0xb8db4dc (0xd48e bytes) 2024-09-13T04:58:49.118 INFO:teuthology.orchestra.run.smithi001.stdout:2516 read 0x6652741 thru 0x6653098 (0x958 bytes) 2024-09-13T04:58:49.118 INFO:teuthology.orchestra.run.smithi001.stdout:2518 write 0xb5af547 thru 0xb5b7402 (0x7ebc bytes) 2024-09-13T04:58:49.127 INFO:teuthology.orchestra.run.smithi001.stdout:2519 read 0xa8b1c61 thru 0xa8c01c4 (0xe564 bytes) 2024-09-13T04:58:49.127 INFO:teuthology.orchestra.run.smithi001.stdout:2521 writesame 0x99d72c4 thru 0x99da042 data_size 0x1(0x2d7f bytes) 2024-09-13T04:58:49.156 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:49.151+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15997 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1314816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135862496000001, 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088423697999999995, 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15997 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026559+0000", 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026558+0000", 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026564+0000", 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4399999999999996e-06 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026580+0000", 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6149000000000001e-05 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026593+0000", 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2723e-05 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114730+0000", 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088137005000000004 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114857+0000", 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012730799999999999 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.114983+0000", 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012595600000000001 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.168 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15998 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135817967999998, 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088737302000000004, 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.169 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15998 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026604+0000", 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026603+0000", 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026608+0000", 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7440000000000004e-06 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026626+0000", 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8114999999999999e-05 2024-09-13T04:58:49.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026638+0000", 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1976e-05 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115046+0000", 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088407311000000002 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115180+0000", 2024-09-13T04:58:49.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013447700000000001 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115341+0000", 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000160676 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15999 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:49.173 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135771773999998, 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089223035000000006, 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15999 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:49.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026650+0000", 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026649+0000", 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026655+0000", 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9560000000000002e-06 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026671+0000", 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026684+0000", 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.273e-05 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115423+0000", 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.088738368999999997 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115651+0000", 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022799699999999999 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115873+0000", 2024-09-13T04:58:49.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000222337 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16000 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135725815999997, 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089511479000000005, 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.178 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16000 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026696+0000", 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026695+0000", 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026701+0000", 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9000000000000003e-06 2024-09-13T04:58:49.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026717+0000", 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6379e-05 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026731+0000", 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4039000000000001e-05 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115965+0000", 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089233554000000007 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116078+0000", 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000113655 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116207+0000", 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128942 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.182 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16001 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1478656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135632850999997, 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089757132000000003, 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16001 2024-09-13T04:58:49.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026789+0000", 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026788+0000", 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026794+0000", 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8059999999999997e-06 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026827+0000", 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3002e-05 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026842+0000", 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4516e-05 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116278+0000", 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089435635999999999 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116414+0000", 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000135919 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116546+0000", 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013235300000000001 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16002 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1519616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.187 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135565884000002, 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089990789000000002, 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16002 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:49.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026856+0000", 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026854+0000", 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026862+0000", 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5789999999999998e-06 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026883+0000", 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1892e-05 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026897+0000", 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.326e-05 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116609+0000", 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089712560999999996 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116708+0000", 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8787000000000001e-05 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116846+0000", 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013842999999999999 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.191 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16003 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1560576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135513256999999, 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090223747000000007, 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16003 2024-09-13T04:58:49.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026908+0000", 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026907+0000", 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026913+0000", 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5770000000000001e-06 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026930+0000", 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7065999999999999e-05 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026943+0000", 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3249e-05 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116903+0000", 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089959340999999998 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116992+0000", 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9690000000000004e-05 2024-09-13T04:58:49.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117132+0000", 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013984800000000001 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16004 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1601536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135454054, 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090483628999999996, 2024-09-13T04:58:49.196 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16004 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026968+0000", 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026967+0000", 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.026978+0000", 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1219e-05 2024-09-13T04:58:49.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027011+0000", 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3222000000000001e-05 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027025+0000", 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3995999999999999e-05 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117203+0000", 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090178122999999999 2024-09-13T04:58:49.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117287+0000", 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3721999999999996e-05 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117451+0000", 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016434300000000001 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16005 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135383163999997, 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090781484999999995, 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16005 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027038+0000", 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027037+0000", 2024-09-13T04:58:49.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027044+0000", 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4640000000000003e-06 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027061+0000", 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6949e-05 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027074+0000", 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3422e-05 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117547+0000", 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.09047318 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117682+0000", 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000134602 2024-09-13T04:58:49.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117820+0000", 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013803200000000001 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16006 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1683456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135334063000002, 2024-09-13T04:58:49.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091107120999999999, 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16006 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027088+0000", 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027086+0000", 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027092+0000", 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8190000000000003e-06 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027109+0000", 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6330000000000001e-05 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027119+0000", 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0132999999999999e-05 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117876+0000", 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.090756823 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118024+0000", 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000148758 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118195+0000", 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00017021500000000001 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16007 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1724416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135295779000003, 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091267529999999999, 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16007 2024-09-13T04:58:49.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027126+0000", 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027125+0000", 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027129+0000", 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5599999999999998e-06 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027137+0000", 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7129999999999994e-06 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027144+0000", 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8329999999999996e-06 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118240+0000", 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091096166000000006 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118313+0000", 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000005e-05 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118393+0000", 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0680999999999995e-05 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16008 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1765376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135271273999997, 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091580432000000003, 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.214 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16008 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027150+0000", 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027153+0000", 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3500000000000001e-06 2024-09-13T04:58:49.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027161+0000", 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1799999999999996e-06 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027168+0000", 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5919999999999997e-06 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118454+0000", 2024-09-13T04:58:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091286415999999995 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118592+0000", 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013743699999999999 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118731+0000", 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013906300000000001 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16009 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135243381000002, 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091763040000000004, 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16009 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.219 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027178+0000", 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027177+0000", 2024-09-13T04:58:49.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027181+0000", 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7160000000000001e-06 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027190+0000", 2024-09-13T04:58:49.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4630000000000001e-06 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027196+0000", 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6429999999999998e-06 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118776+0000", 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091579259999999996 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118855+0000", 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9695000000000002e-05 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118941+0000", 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6013000000000005e-05 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16010 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1847296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.223 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135219515999999, 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091942769999999993, 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16010 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:49.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027202+0000", 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027201+0000", 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027205+0000", 2024-09-13T04:58:49.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.213e-06 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027213+0000", 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2460000000000003e-06 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027220+0000", 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9970000000000001e-06 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118986+0000", 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091766174000000006 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119056+0000", 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9988999999999997e-05 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119145+0000", 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8751000000000005e-05 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16011 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1888256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135195541000002, 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130156000000005, 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16011 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027226+0000", 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027225+0000", 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027229+0000", 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1640000000000001e-06 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027237+0000", 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3299999999999999e-06 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027243+0000", 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4620000000000001e-06 2024-09-13T04:58:49.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119188+0000", 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.091944247000000007 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119271+0000", 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3225000000000004e-05 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119356+0000", 2024-09-13T04:58:49.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5325e-05 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16012 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135171939000003, 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092295000000000002, 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16012 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027250+0000", 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027249+0000", 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027252+0000", 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0970000000000002e-06 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027260+0000", 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2959999999999999e-06 2024-09-13T04:58:49.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027267+0000", 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6059999999999999e-06 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119398+0000", 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092130904 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119466+0000", 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8460000000000005e-05 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119545+0000", 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8230000000000001e-05 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16013 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:49.236 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.13514868, 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092535714000000005, 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16013 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027273+0000", 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027272+0000", 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027275+0000", 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1130000000000001e-06 2024-09-13T04:58:49.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027284+0000", 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2430000000000005e-06 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027290+0000", 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4999999999999996e-06 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119588+0000", 2024-09-13T04:58:49.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092297660000000004 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119700+0000", 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000112526 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119809+0000", 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108262 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16014 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2011136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135125618000004, 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092839985999999999, 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16014 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027296+0000", 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027295+0000", 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027298+0000", 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1329999999999999e-06 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027307+0000", 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4859999999999997e-06 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027314+0000", 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5799999999999997e-06 2024-09-13T04:58:49.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.119880+0000", 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092566792999999994 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120027+0000", 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014627800000000001 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120136+0000", 2024-09-13T04:58:49.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109329 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16015 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2052096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135102203000002, 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093043843000000001, 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16015 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027319+0000", 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027322+0000", 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.146e-06 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027330+0000", 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.524e-06 2024-09-13T04:58:49.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027337+0000", 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4659999999999996e-06 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120194+0000", 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092856578999999995 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120268+0000", 2024-09-13T04:58:49.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4470999999999999e-05 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120363+0000", 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5253000000000004e-05 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:16016 3.1b 3:dd59f0a6:::rbd_data.112af6c12117.0000000000000001:6 [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.135078684, 2024-09-13T04:58:49.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093219623000000001, 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 16016 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027343+0000", 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027342+0000", 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027352+0000", 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0000000000000001e-05 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027361+0000", 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7090000000000008e-06 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.027368+0000", 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629999999999997e-06 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120393+0000", 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.093025262999999997 2024-09-13T04:58:49.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120471+0000", 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7688000000000006e-05 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.120563+0000", 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2003000000000007e-05 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.253 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.254 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:49.254 INFO:teuthology.orchestra.run.smithi001.stdout:2522 trunc from 0xd7697db to 0x153ff81 2024-09-13T04:58:49.256 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:49.257 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:49.255+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:49.330 INFO:teuthology.orchestra.run.smithi001.stdout:2523 read 0x95d821 thru 0x960bfd (0x33dd bytes) 2024-09-13T04:58:49.331 INFO:teuthology.orchestra.run.smithi001.stdout:2524 punch from 0xf79e13 to 0xf87090, (0xd27d bytes) 2024-09-13T04:58:49.332 INFO:teuthology.orchestra.run.smithi001.stdout:2525 punch from 0x5ac5eb to 0x5b9583, (0xcf98 bytes) 2024-09-13T04:58:49.353 INFO:teuthology.orchestra.run.smithi001.stdout:2526 writesame 0x654ea8c thru 0x6557993 data_size 0x2e(0x8f08 bytes) 2024-09-13T04:58:49.359 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:49.355+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:49.368 INFO:teuthology.orchestra.run.smithi001.stdout:2528 trunc from 0x6557994 to 0xd253aa9 2024-09-13T04:58:49.378 INFO:teuthology.orchestra.run.smithi001.stdout:2529 punch from 0x3a0b5ea to 0x3a18658, (0xd06e bytes) 2024-09-13T04:58:49.378 INFO:teuthology.orchestra.run.smithi001.stdout:2530 writesame 0x946265f thru 0x947021c data_size 0xb(0xdbbe bytes) 2024-09-13T04:58:49.388 INFO:teuthology.orchestra.run.smithi001.stdout:2531 write 0x448e52d thru 0x449d80b (0xf2df bytes) 2024-09-13T04:58:49.397 INFO:teuthology.orchestra.run.smithi001.stdout:2532 read 0x39e2086 thru 0x39e50cc (0x3047 bytes) 2024-09-13T04:58:49.397 INFO:teuthology.orchestra.run.smithi001.stdout:2533 punch from 0x1c2a673 to 0x1c321b1, (0x7b3e bytes) 2024-09-13T04:58:49.398 INFO:teuthology.orchestra.run.smithi001.stdout:2535 read 0x49d248e thru 0x49d869c (0x620f bytes) 2024-09-13T04:58:49.398 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.398 INFO:teuthology.orchestra.run.smithi001.stdout:2538 punch from 0x9ba8c95 to 0x9badbf9, (0x4f64 bytes) 2024-09-13T04:58:49.398 INFO:teuthology.orchestra.run.smithi001.stdout:2539 writesame 0x83bb141 thru 0x83bec78 data_size 0x14(0x3b38 bytes) 2024-09-13T04:58:49.404 INFO:teuthology.orchestra.run.smithi001.stdout:2540 write 0x37b98b9 thru 0x37c297f (0x90c7 bytes) 2024-09-13T04:58:49.417 INFO:teuthology.orchestra.run.smithi001.stdout:2541 write 0x5567cb4 thru 0x556db78 (0x5ec5 bytes) 2024-09-13T04:58:49.424 INFO:teuthology.orchestra.run.smithi001.stdout:2544 writesame 0x1dd743d thru 0x1ddbd9c data_size 0x10(0x4960 bytes) 2024-09-13T04:58:49.429 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:49.429 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:49.429 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:49.429 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:49.429 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.430 INFO:teuthology.orchestra.run.smithi001.stdout:2546 read 0x9463e12 thru 0x946e6f0 (0xa8df bytes) 2024-09-13T04:58:49.443 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:49.448 INFO:teuthology.orchestra.run.smithi001.stdout:2547 trunc from 0xd253aa9 to 0x530f54e 2024-09-13T04:58:49.459 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:49.455+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:49.494 INFO:teuthology.orchestra.run.smithi001.stdout:2548 write 0x752d8d8 thru 0x7533ce2 (0x640b bytes) 2024-09-13T04:58:49.502 INFO:teuthology.orchestra.run.smithi001.stdout:2551 writesame 0x49d290c thru 0x49dfc5b data_size 0x62(0xd350 bytes) 2024-09-13T04:58:49.539 INFO:teuthology.orchestra.run.smithi001.stdout:2552 writesame 0xd5c5f9c thru 0xd5d2f10 data_size 0x3f(0xcf75 bytes) 2024-09-13T04:58:49.540 INFO:teuthology.orchestra.run.smithi001.stdout:2554 write 0x77e8286 thru 0x77eff0d (0x7c88 bytes) 2024-09-13T04:58:49.551 INFO:teuthology.orchestra.run.smithi001.stdout:2555 punch from 0x13f217d to 0x13f8cb0, (0x6b33 bytes) 2024-09-13T04:58:49.551 INFO:teuthology.orchestra.run.smithi001.stdout:2556 writesame 0x79dc658 thru 0x79e494e data_size 0x1(0x82f7 bytes) 2024-09-13T04:58:49.560 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:49.555+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:49.560 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:49.560 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:49.560 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:49.560 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:49.561 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:49.574 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:49.613 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.613 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.614 INFO:teuthology.orchestra.run.smithi001.stdout:2560 read 0x9eeb93b thru 0x9ef3c50 (0x8316 bytes) 2024-09-13T04:58:49.614 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.614 INFO:teuthology.orchestra.run.smithi001.stdout:2567 write 0x2634b2f thru 0x2635fc2 (0x1494 bytes) 2024-09-13T04:58:49.621 INFO:teuthology.orchestra.run.smithi001.stdout:2568 write 0x1748863 thru 0x17548da (0xc078 bytes) 2024-09-13T04:58:49.642 INFO:teuthology.orchestra.run.smithi001.stdout:2569 writesame 0xd73dead thru 0xd74a7fd data_size 0x29(0xc951 bytes) 2024-09-13T04:58:49.642 INFO:teuthology.orchestra.run.smithi001.stdout:2570 read 0x34fb718 thru 0x350467e (0x8f67 bytes) 2024-09-13T04:58:49.642 INFO:teuthology.orchestra.run.smithi001.stdout:2571 punch from 0xb5207d2 to 0xb52e5f3, (0xde21 bytes) 2024-09-13T04:58:49.643 INFO:teuthology.orchestra.run.smithi001.stdout:2572 writesame 0x4c92728 thru 0x4c9f8f2 data_size 0x1(0xd1cb bytes) 2024-09-13T04:58:49.660 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:49.655+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:49.723 INFO:teuthology.orchestra.run.smithi001.stdout:2573 read 0x99bab1c thru 0x99ca391 (0xf876 bytes) 2024-09-13T04:58:49.726 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.745 INFO:teuthology.orchestra.run.smithi001.stdout:2575 writesame 0xdb0e0c6 thru 0xdb1c851 data_size 0x92(0xe78c bytes) 2024-09-13T04:58:49.745 INFO:teuthology.orchestra.run.smithi001.stdout:2576 read 0xbc2baa4 thru 0xbc2ef12 (0x346f bytes) 2024-09-13T04:58:49.745 INFO:teuthology.orchestra.run.smithi001.stdout:2577 write 0x746b7ab thru 0x746f869 (0x40bf bytes) 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.708911419000003, 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:58:49.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:58:49.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:58:49.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:58:49.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:58:49.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.751 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.708865781999997, 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:49.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:58:49.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:58:49.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:58:49.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.757 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.708817697999997, 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:58:49.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.708721615999998, 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:58:49.766 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.708670245, 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:58:49.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.708624190999998, 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:58:49.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.708574130000002, 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.776 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.777 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.777 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:58:49.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.777 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:58:49.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:49.781 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.59115894, 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:58:49.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:58:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.787 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499730556999999, 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.788 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.792 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499706212, 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:58:49.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499624859000001, 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.797 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.798 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.798 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.798 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.798 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:58:49.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.798 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:58:49.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499555329, 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:58:49.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.806 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499528052999999, 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.807 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:58:49.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:58:49.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:58:49.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:58:49.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.812 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499429608, 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:49.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:49.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:49.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:58:49.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:58:49.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:58:49.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499360040999999, 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.819 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:49.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:58:49.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499289567000002, 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:49.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:49.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:58:49.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:58:49.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:58:49.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:58:49.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:58:49.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:58:49.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:58:49.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:58:49.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:58:49.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:58:49.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:58:49.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:58:49.832 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.832 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.832 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.833 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.833 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:49.833 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499263882000001, 2024-09-13T04:58:49.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:58:49.834 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.834 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.834 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:58:49.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.835 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:49.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:49.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:49.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:58:49.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:58:49.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:58:49.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:58:49.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:58:49.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:58:49.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:58:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:58:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:58:49.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:58:49.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:58:49.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:58:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499228693999999, 2024-09-13T04:58:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:58:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:58:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.845 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:58:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:58:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:58:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:58:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:58:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:58:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:58:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:58:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:58:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:58:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:58:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:58:49.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:58:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499204177999999, 2024-09-13T04:58:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:58:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.855 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:58:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:49.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:58:49.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.861 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.499134841, 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:49.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:49.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:58:49.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:58:49.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.866 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:49.866 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:49.866 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:49.867 INFO:teuthology.orchestra.run.smithi001.stdout:2578 read 0xbc8a77c thru 0xbc8e87a (0x40ff bytes) 2024-09-13T04:58:49.867 INFO:teuthology.orchestra.run.smithi001.stdout:2579 trunc from 0xdb1c852 to 0x122648a 2024-09-13T04:58:49.868 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:49.868 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:49.863+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:49.881 INFO:teuthology.orchestra.run.smithi001.stdout:2580 write 0xe7cd655 thru 0xe7da17c (0xcb28 bytes) 2024-09-13T04:58:49.889 INFO:teuthology.orchestra.run.smithi001.stdout:2581 write 0x3801d83 thru 0x3803110 (0x138e bytes) 2024-09-13T04:58:49.905 INFO:teuthology.orchestra.run.smithi001.stdout:2582 trunc from 0xe7da17d to 0xab01b0d 2024-09-13T04:58:49.927 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.928 INFO:teuthology.orchestra.run.smithi001.stdout:2584 writesame 0x40106e8 thru 0x4016ab8 data_size 0x17(0x63d1 bytes) 2024-09-13T04:58:49.936 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:49.946 INFO:teuthology.orchestra.run.smithi001.stdout:2586 trunc from 0xab01b0d to 0xe7dc2ab 2024-09-13T04:58:49.960 INFO:teuthology.orchestra.run.smithi001.stdout:2588 flatten 2024-09-13T04:58:49.971 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:49.967+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:50.070 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:50.067+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:50.078 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:50.079 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:50.079 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:50.099 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:50.188 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:50.187+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:50.227 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:50.272 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:50.267+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:50.287 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:50.287 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:50.288 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:50.288 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:50.302 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:50.392 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:50.391+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:50.409 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:50.409 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:50.425 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:50.425 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:50.447 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:50.464 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:50.464 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:50.474 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:50.471+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:50.484 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:50.484 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:50.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:50.505 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:50.505 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:50.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:50.505 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:50.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.505 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.505 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:58:50.506 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703384833000001, 2024-09-13T04:58:50.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:58:50.506 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.506 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.506 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.506 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:58:50.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:58:50.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:58:50.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.511 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.511 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.511 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.511 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703350578999999, 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:58:50.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:58:50.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703319678, 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.517 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:58:50.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:58:50.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:58:50.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:58:50.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:58:50.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.522 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703267796999999, 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.527 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703235994, 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:58:50.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:58:50.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:58:50.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:58:50.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:58:50.532 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.532 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.532 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.533 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.533 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:58:50.533 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703202041000001, 2024-09-13T04:58:50.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:58:50.533 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.533 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:58:50.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703154304999998, 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.538 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:58:50.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:58:50.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:58:50.543 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.543 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.543 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.543 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.543 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:58:50.544 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703114743, 2024-09-13T04:58:50.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:58:50.544 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.544 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.544 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.545 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:58:50.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.546 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:58:50.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:58:50.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:58:50.550 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703067049000001, 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.551 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:58:50.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:58:50.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:58:50.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.556 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.703003966000001, 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:58:50.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702967531999999, 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.561 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:58:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:58:50.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702933608999999, 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.566 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:58:50.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:58:50.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702865396, 2024-09-13T04:58:50.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:58:50.572 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.572 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.572 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:58:50.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:58:50.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.577 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.70283169, 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:58:50.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:58:50.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:58:50.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702781498, 2024-09-13T04:58:50.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:58:50.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:58:50.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:58:50.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:58:50.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702475464999999, 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.587 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:58:50.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.591 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702426263, 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:58:50.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:58:50.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702378149000001, 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.596 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:58:50.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:58:50.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:58:50.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:58:50.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:58:50.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702326672000002, 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.602 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:58:50.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:58:50.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:58:50.606 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 26.702291591000002, 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:58:50.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:50.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.610 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.611 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:50.611 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:50.611 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:50.612 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:50.612 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:50.613 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:50.613 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:50.614 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:50.615 INFO:teuthology.orchestra.run.smithi001.stdout:2589 punch from 0xa15d3df to 0xa164557, (0x7178 bytes) 2024-09-13T04:58:50.615 INFO:teuthology.orchestra.run.smithi001.stdout:2590 write 0xcbfb556 thru 0xcc06754 (0xb1ff bytes) 2024-09-13T04:58:50.615 INFO:teuthology.orchestra.run.smithi001.stdout:2592 write 0x4fed350 thru 0x4ffa8b6 (0xd567 bytes) 2024-09-13T04:58:50.615 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:50.607+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:50.615 INFO:teuthology.orchestra.run.smithi001.stdout:2593 writesame 0xccdd790 thru 0xcceafa7 data_size 0x3c(0xd818 bytes) 2024-09-13T04:58:50.619 INFO:teuthology.orchestra.run.smithi001.stdout:2594 write 0x7bb1400 thru 0x7bb59c6 (0x45c7 bytes) 2024-09-13T04:58:50.632 INFO:teuthology.orchestra.run.smithi001.stdout:2595 trunc from 0xe7dc2ab to 0xb143dba 2024-09-13T04:58:50.646 INFO:teuthology.orchestra.run.smithi001.stdout:2597 read 0x8808f21 thru 0x881844d (0xf52d bytes) 2024-09-13T04:58:50.646 INFO:teuthology.orchestra.run.smithi001.stdout:2598 punch from 0x65e3b7f to 0x65e3f5f, (0x3e0 bytes) 2024-09-13T04:58:50.660 INFO:teuthology.orchestra.run.smithi001.stdout:2599 write 0xbc11731 thru 0xbc1ae8e (0x975e bytes) 2024-09-13T04:58:50.665 INFO:teuthology.orchestra.run.smithi001.stdout:2600 read 0x9ba922b thru 0x9ba9314 (0xea bytes) 2024-09-13T04:58:50.674 INFO:teuthology.orchestra.run.smithi001.stdout:2601 trunc from 0xbc1ae8f to 0x81b8799 2024-09-13T04:58:50.713 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:50.711+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:50.723 INFO:teuthology.orchestra.run.smithi001.stdout:2603 writesame 0xc0d35b3 thru 0xc0d74d7 data_size 0x35(0x3f25 bytes) 2024-09-13T04:58:50.723 INFO:teuthology.orchestra.run.smithi001.stdout:2604 read 0x83c9d97 thru 0x83d50fa (0xb364 bytes) 2024-09-13T04:58:50.723 INFO:teuthology.orchestra.run.smithi001.stdout:2605 write 0x9c7a34a thru 0x9c7efcd (0x4c84 bytes) 2024-09-13T04:58:50.728 INFO:teuthology.orchestra.run.smithi001.stdout:2606 punch from 0x6fd1bc8 to 0x6fdb0ff, (0x9537 bytes) 2024-09-13T04:58:50.728 INFO:teuthology.orchestra.run.smithi001.stdout:2607 write 0xa28d8c9 thru 0xa28f12f (0x1867 bytes) 2024-09-13T04:58:50.736 INFO:teuthology.orchestra.run.smithi001.stdout:2608 read 0x3b99f9 thru 0x3c60b5 (0xc6bd bytes) 2024-09-13T04:58:50.737 INFO:teuthology.orchestra.run.smithi001.stdout:2609 write 0x14a459d thru 0x14a82c3 (0x3d27 bytes) 2024-09-13T04:58:50.742 INFO:teuthology.orchestra.run.smithi001.stdout:2610 punch from 0x46f5176 to 0x46f9b55, (0x49df bytes) 2024-09-13T04:58:50.742 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:50.742 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:50.742 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:50.764 INFO:teuthology.orchestra.run.smithi001.stdout:2617 trunc from 0xc0d74d8 to 0x3dbe744 2024-09-13T04:58:50.787 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:50.787 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:50.799 INFO:teuthology.orchestra.run.smithi001.stdout:2618 trunc from 0x3dbe744 to 0x54cb615 2024-09-13T04:58:50.801 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:50.813 INFO:teuthology.orchestra.run.smithi001.stdout:2619 punch from 0x4c24799 to 0x4c2c630, (0x7e97 bytes) 2024-09-13T04:58:50.838 INFO:teuthology.orchestra.run.smithi001.stdout:2620 writesame 0x91a0196 thru 0x91a837a data_size 0x1(0x81e5 bytes) 2024-09-13T04:58:50.838 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:50.838 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:50.844 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:50.843+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:50.849 INFO:teuthology.orchestra.run.smithi001.stdout:2624 trunc from 0x91a837b to 0x4a03f7d 2024-09-13T04:58:50.863 INFO:teuthology.orchestra.run.smithi001.stdout:2626 read 0x484e72d thru 0x48591ae (0xaa82 bytes) 2024-09-13T04:58:50.876 INFO:teuthology.orchestra.run.smithi001.stdout:2627 write 0x4d29c8f thru 0x4d2bfec (0x235e bytes) 2024-09-13T04:58:50.883 INFO:teuthology.orchestra.run.smithi001.stdout:2628 write 0x2ac4772 thru 0x2accd7b (0x860a bytes) 2024-09-13T04:58:50.887 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:58:50.894 INFO:teuthology.orchestra.run.smithi001.stdout:2629 punch from 0x547d9e to 0x55182e, (0x9a90 bytes) 2024-09-13T04:58:50.895 INFO:teuthology.orchestra.run.smithi001.stdout:2630 writesame 0x40505d0 thru 0x405dc60 data_size 0x3b(0xd691 bytes) 2024-09-13T04:58:50.903 INFO:teuthology.orchestra.run.smithi001.stdout:2631 punch from 0x3802de2 to 0x3808965, (0x5b83 bytes) 2024-09-13T04:58:50.915 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:50.911+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:50.935 INFO:teuthology.orchestra.run.smithi001.stdout:2632 writesame 0xea0e0f3 thru 0xea116c0 data_size 0x2(0x35ce bytes) 2024-09-13T04:58:50.936 INFO:teuthology.orchestra.run.smithi001.stdout:2633 write 0x369f3f5 thru 0x36a8d02 (0x990e bytes) 2024-09-13T04:58:50.963 INFO:teuthology.orchestra.run.smithi001.stdout:2634 trunc from 0xea116c1 to 0x4118c45 2024-09-13T04:58:50.969 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:50.969 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:50.984 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:2635 read 0x137bfd thru 0x1390c6 (0x14ca bytes) 2024-09-13T04:58:50.994 INFO:teuthology.orchestra.run.smithi001.stdout:2636 trunc from 0x4118c45 to 0x94c0f5a 2024-09-13T04:58:51.005 INFO:teuthology.orchestra.run.smithi001.stdout:2638 punch from 0x1ae16da to 0x1ae1fb4, (0x8da bytes) 2024-09-13T04:58:51.006 INFO:teuthology.orchestra.run.smithi001.stdout:2640 writesame 0x8cb8435 thru 0x8cb8600 data_size 0xa(0x1cc bytes) 2024-09-13T04:58:51.011 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.024 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:51.023+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:51.030 INFO:teuthology.orchestra.run.smithi001.stdout:2643 write 0xd0fc138 thru 0xd1079d4 (0xb89d bytes) 2024-09-13T04:58:51.038 INFO:teuthology.orchestra.run.smithi001.stdout:2644 punch from 0xbc99973 to 0xbca3e46, (0xa4d3 bytes) 2024-09-13T04:58:51.039 INFO:teuthology.orchestra.run.smithi001.stdout:2645 read 0xb08b9df thru 0xb091049 (0x566b bytes) 2024-09-13T04:58:51.039 INFO:teuthology.orchestra.run.smithi001.stdout:2646 punch from 0x70d3b55 to 0x70ddf8b, (0xa436 bytes) 2024-09-13T04:58:51.039 INFO:teuthology.orchestra.run.smithi001.stdout:2647 write 0x41fb673 thru 0x4205625 (0x9fb3 bytes) 2024-09-13T04:58:51.046 INFO:teuthology.orchestra.run.smithi001.stdout:2650 punch from 0x5ee156a to 0x5ee39cc, (0x2462 bytes) 2024-09-13T04:58:51.046 INFO:teuthology.orchestra.run.smithi001.stdout:2651 punch from 0x7bcc8f2 to 0x7bd94ed, (0xcbfb bytes) 2024-09-13T04:58:51.047 INFO:teuthology.orchestra.run.smithi001.stdout:2652 writesame 0xa7498f9 thru 0xa757c2f data_size 0x45(0xe337 bytes) 2024-09-13T04:58:51.051 INFO:teuthology.orchestra.run.smithi001.stdout:2655 write 0x646795 thru 0x64792d (0x1199 bytes) 2024-09-13T04:58:51.054 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.062 INFO:teuthology.orchestra.run.smithi001.stdout:2658 trunc from 0xd1079d5 to 0x8aa966d 2024-09-13T04:58:51.076 INFO:teuthology.orchestra.run.smithi001.stdout:2660 read 0x1be9da1 thru 0x1bf1a7e (0x7cde bytes) 2024-09-13T04:58:51.085 INFO:teuthology.orchestra.run.smithi001.stdout:2661 write 0x92c2ef4 thru 0x92c3634 (0x741 bytes) 2024-09-13T04:58:51.089 INFO:teuthology.orchestra.run.smithi001.stdout:2663 read 0x3baacf9 thru 0x3bb13fa (0x6702 bytes) 2024-09-13T04:58:51.089 INFO:teuthology.orchestra.run.smithi001.stdout:2665 read 0x34b5a25 thru 0x34bd828 (0x7e04 bytes) 2024-09-13T04:58:51.089 INFO:teuthology.orchestra.run.smithi001.stdout:2666 write 0x27a014f thru 0x27a24e2 (0x2394 bytes) 2024-09-13T04:58:51.093 INFO:teuthology.orchestra.run.smithi001.stdout:2667 punch from 0x4d97909 to 0x4da4959, (0xd050 bytes) 2024-09-13T04:58:51.093 INFO:teuthology.orchestra.run.smithi001.stdout:2668 write 0x574a4e5 thru 0x57592ec (0xee08 bytes) 2024-09-13T04:58:51.098 INFO:teuthology.orchestra.run.smithi001.stdout:2669 writesame 0x5f067fa thru 0x5f10c7f data_size 0x1(0xa486 bytes) 2024-09-13T04:58:51.116 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:51.111+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:51.153 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:51.154 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:51.163 INFO:teuthology.orchestra.run.smithi001.stdout:2670 write 0xad06fe3 thru 0xad11f9e (0xafbc bytes) 2024-09-13T04:58:51.168 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:51.169 INFO:teuthology.orchestra.run.smithi001.stdout:2671 write 0x6a5f82b thru 0x6a606ec (0xec2 bytes) 2024-09-13T04:58:51.178 INFO:teuthology.orchestra.run.smithi001.stdout:2672 read 0x7a32967 thru 0x7a39885 (0x6f1f bytes) 2024-09-13T04:58:51.178 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.184 INFO:teuthology.orchestra.run.smithi001.stdout:2674 trunc from 0xad11f9f to 0x7821fb0 2024-09-13T04:58:51.203 INFO:teuthology.orchestra.run.smithi001.stdout:2676 writesame 0x3403d6c thru 0x340d396 data_size 0x1(0x962b bytes) 2024-09-13T04:58:51.216 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:51.215+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:51.239 INFO:teuthology.orchestra.run.smithi001.stdout:2677 read 0x57b2a18 thru 0x57c191e (0xef07 bytes) 2024-09-13T04:58:51.242 INFO:teuthology.orchestra.run.smithi001.stdout:2678 write 0xa4d273 thru 0xa56905 (0x9693 bytes) 2024-09-13T04:58:51.275 INFO:teuthology.orchestra.run.smithi001.stdout:2679 writesame 0xe8b7090 thru 0xe8c2291 data_size 0xba(0xb202 bytes) 2024-09-13T04:58:51.276 INFO:teuthology.orchestra.run.smithi001.stdout:2681 read 0x4487b79 thru 0x449030c (0x8794 bytes) 2024-09-13T04:58:51.276 INFO:teuthology.orchestra.run.smithi001.stdout:2682 read 0x11add72 thru 0x11b50b3 (0x7342 bytes) 2024-09-13T04:58:51.287 INFO:teuthology.orchestra.run.smithi001.stdout:2684 trunc from 0xe8c2292 to 0x851d3c3 2024-09-13T04:58:51.305 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:51.305 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:51.306 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.317 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:51.315+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:51.325 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:51.325 INFO:teuthology.orchestra.run.smithi001.stdout:2690 write 0xbe06a9e thru 0xbe1634e (0xf8b1 bytes) 2024-09-13T04:58:51.331 INFO:teuthology.orchestra.run.smithi001.stdout:2691 write 0x533cd7b thru 0x5340897 (0x3b1d bytes) 2024-09-13T04:58:51.339 INFO:teuthology.orchestra.run.smithi001.stdout:2692 write 0x82f30d5 thru 0x82f8291 (0x51bd bytes) 2024-09-13T04:58:51.347 INFO:teuthology.orchestra.run.smithi001.stdout:2693 write 0x448088e thru 0x448e16e (0xd8e1 bytes) 2024-09-13T04:58:51.359 INFO:teuthology.orchestra.run.smithi001.stdout:2694 writesame 0x53fe17f thru 0x54095f6 data_size 0x1(0xb478 bytes) 2024-09-13T04:58:51.391 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.391 INFO:teuthology.orchestra.run.smithi001.stdout:2697 write 0x511edf3 thru 0x5121cd4 (0x2ee2 bytes) 2024-09-13T04:58:51.400 INFO:teuthology.orchestra.run.smithi001.stdout:2698 writesame 0x27371a1 thru 0x273efc5 data_size 0x1(0x7e25 bytes) 2024-09-13T04:58:51.418 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:51.415+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:51.420 INFO:teuthology.orchestra.run.smithi001.stdout:2699 read 0x83cbc53 thru 0x83d9eeb (0xe299 bytes) 2024-09-13T04:58:51.421 INFO:teuthology.orchestra.run.smithi001.stdout:2700 writesame 0x2d0407f thru 0x2d125d3 data_size 0x1(0xe555 bytes) 2024-09-13T04:58:51.461 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:51.461 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:51.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:51.478 INFO:teuthology.orchestra.run.smithi001.stdout:2701 write 0xb2b4983 thru 0xb2b6582 (0x1c00 bytes) 2024-09-13T04:58:51.490 INFO:teuthology.orchestra.run.smithi001.stdout:2702 write 0x8440648 thru 0x8445bf2 (0x55ab bytes) 2024-09-13T04:58:51.495 INFO:teuthology.orchestra.run.smithi001.stdout:2703 writesame 0x5523040 thru 0x552e93e data_size 0x1(0xb8ff bytes) 2024-09-13T04:58:51.519 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:51.515+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:51.578 INFO:teuthology.orchestra.run.smithi001.stdout:2704 write 0xdc4ab23 thru 0xdc593c8 (0xe8a6 bytes) 2024-09-13T04:58:51.600 INFO:teuthology.orchestra.run.smithi001.stdout:2706 writesame 0xe3f39f7 thru 0xe3ff2dd data_size 0x1(0xb8e7 bytes) 2024-09-13T04:58:51.600 INFO:teuthology.orchestra.run.smithi001.stdout:2707 read 0x9c6a9cf thru 0x9c6f571 (0x4ba3 bytes) 2024-09-13T04:58:51.601 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:51.601 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:51.601 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.613 INFO:teuthology.orchestra.run.smithi001.stdout:2709 trunc from 0xe3ff2de to 0x8c6b869 2024-09-13T04:58:51.621 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:51.621 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:51.615+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:51.651 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.651 INFO:teuthology.orchestra.run.smithi001.stdout:2712 read 0x1c12e45 thru 0x1c21e75 (0xf031 bytes) 2024-09-13T04:58:51.653 INFO:teuthology.orchestra.run.smithi001.stdout:2713 writesame 0x387a40d thru 0x387e138 data_size 0xf(0x3d2c bytes) 2024-09-13T04:58:51.677 INFO:teuthology.orchestra.run.smithi001.stdout:2714 write 0xbf3ff8a thru 0xbf4f6ab (0xf722 bytes) 2024-09-13T04:58:51.691 INFO:teuthology.orchestra.run.smithi001.stdout:2716 trunc from 0xbf4f6ac to 0xe909156 2024-09-13T04:58:51.704 INFO:teuthology.orchestra.run.smithi001.stdout:2721 punch from 0x67a92d6 to 0x67b649a, (0xd1c4 bytes) 2024-09-13T04:58:51.704 INFO:teuthology.orchestra.run.smithi001.stdout:2722 read 0xb6356c6 thru 0xb6438fb (0xe236 bytes) 2024-09-13T04:58:51.704 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.704 INFO:teuthology.orchestra.run.smithi001.stdout:2726 writesame 0x3c242d5 thru 0x3c2d71c data_size 0x49(0x9448 bytes) 2024-09-13T04:58:51.709 INFO:teuthology.orchestra.run.smithi001.stdout:2730 write 0x816ff5d thru 0x8171521 (0x15c5 bytes) 2024-09-13T04:58:51.715 INFO:teuthology.orchestra.run.smithi001.stdout:2731 punch from 0xc9d3ea to 0xca7c0e, (0xa824 bytes) 2024-09-13T04:58:51.721 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:51.715+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:51.737 INFO:teuthology.orchestra.run.smithi001.stdout:2732 trunc from 0xe909156 to 0x4c5fb40 2024-09-13T04:58:51.772 INFO:teuthology.orchestra.run.smithi001.stdout:2734 punch from 0x4608b45 to 0x46094d4, (0x98f bytes) 2024-09-13T04:58:51.795 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:51.795 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:51.801 INFO:teuthology.orchestra.run.smithi001.stdout:2735 writesame 0xe11c1e4 thru 0xe11db76 data_size 0x1(0x1993 bytes) 2024-09-13T04:58:51.801 INFO:teuthology.orchestra.run.smithi001.stdout:2736 read 0x71761aa thru 0x717e3ed (0x8244 bytes) 2024-09-13T04:58:51.801 INFO:teuthology.orchestra.run.smithi001.stdout:2738 writesame 0xbb5af41 thru 0xbb64da5 data_size 0x29(0x9e65 bytes) 2024-09-13T04:58:51.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:51.811 INFO:teuthology.orchestra.run.smithi001.stdout:2739 write 0x7659e45 thru 0x765e15f (0x431b bytes) 2024-09-13T04:58:51.816 INFO:teuthology.orchestra.run.smithi001.stdout:2740 write 0x23706be thru 0x237f1e7 (0xeb2a bytes) 2024-09-13T04:58:51.821 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:51.819+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:51.823 INFO:teuthology.orchestra.run.smithi001.stdout:2742 punch from 0x6b27ac7 to 0x6b37a32, (0xff6b bytes) 2024-09-13T04:58:51.844 INFO:teuthology.orchestra.run.smithi001.stdout:2743 trunc from 0xe11db77 to 0x4e85853 2024-09-13T04:58:51.867 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.867 INFO:teuthology.orchestra.run.smithi001.stdout:2746 read 0x4c88fc9 thru 0x4c96f71 (0xdfa9 bytes) 2024-09-13T04:58:51.867 INFO:teuthology.orchestra.run.smithi001.stdout:2747 punch from 0x31405d3 to 0x314edbc, (0xe7e9 bytes) 2024-09-13T04:58:51.867 INFO:teuthology.orchestra.run.smithi001.stdout:2749 write 0x14915e0 thru 0x149c7a2 (0xb1c3 bytes) 2024-09-13T04:58:51.886 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:58:51.902 INFO:teuthology.orchestra.run.smithi001.stdout:2750 write 0xca9a8b8 thru 0xcaa04bb (0x5c04 bytes) 2024-09-13T04:58:51.909 INFO:teuthology.orchestra.run.smithi001.stdout:2751 writesame 0x8413452 thru 0x8422ce2 data_size 0x1(0xf891 bytes) 2024-09-13T04:58:51.957 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:51.955+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:51.958 INFO:teuthology.orchestra.run.smithi001.stdout:2752 punch from 0x2dd5b90 to 0x2dd6f34, (0x13a4 bytes) 2024-09-13T04:58:51.958 INFO:teuthology.orchestra.run.smithi001.stdout:2753 write 0xa6ecc11 thru 0xa6f6d73 (0xa163 bytes) 2024-09-13T04:58:51.958 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:51.959 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:51.967 INFO:teuthology.orchestra.run.smithi001.stdout:2756 write 0xa6bc920 thru 0xa6bf86d (0x2f4e bytes) 2024-09-13T04:58:51.973 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:51.973 INFO:teuthology.orchestra.run.smithi001.stdout:2758 read 0x5f178bf thru 0x5f206d7 (0x8e19 bytes) 2024-09-13T04:58:51.973 INFO:teuthology.orchestra.run.smithi001.stdout:2759 read 0x8f07112 thru 0x8f07b10 (0x9ff bytes) 2024-09-13T04:58:51.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:51.981 INFO:teuthology.orchestra.run.smithi001.stdout:2760 trunc from 0xcaa04bc to 0x6dd14f1 2024-09-13T04:58:52.011 INFO:teuthology.orchestra.run.smithi001.stdout:2761 trunc from 0x6dd14f1 to 0xdd2d88d 2024-09-13T04:58:52.022 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:52.022 INFO:teuthology.orchestra.run.smithi001.stdout:2763 write 0x9ea7d58 thru 0x9eb44b4 (0xc75d bytes) 2024-09-13T04:58:52.023 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:52.019+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:52.033 INFO:teuthology.orchestra.run.smithi001.stdout:2764 punch from 0x2cc85e7 to 0x2ccd011, (0x4a2a bytes) 2024-09-13T04:58:52.033 INFO:teuthology.orchestra.run.smithi001.stdout:2766 read 0xbf7095c thru 0xbf7df22 (0xd5c7 bytes) 2024-09-13T04:58:52.033 INFO:teuthology.orchestra.run.smithi001.stdout:2767 read 0x1e7dd6d thru 0x1e7f134 (0x13c8 bytes) 2024-09-13T04:58:52.034 INFO:teuthology.orchestra.run.smithi001.stdout:2770 write 0xc714ccf thru 0xc724b59 (0xfe8b bytes) 2024-09-13T04:58:52.054 INFO:teuthology.orchestra.run.smithi001.stdout:2772 trunc from 0xdd2d88d to 0x62b7ab6 2024-09-13T04:58:52.074 INFO:teuthology.orchestra.run.smithi001.stdout:2773 trunc from 0x62b7ab6 to 0x567f551 2024-09-13T04:58:52.109 INFO:teuthology.orchestra.run.smithi001.stdout:2775 writesame 0xa61d5e3 thru 0xa62a69b data_size 0x9(0xd0b9 bytes) 2024-09-13T04:58:52.109 INFO:teuthology.orchestra.run.smithi001.stdout:2777 read 0x772e5eb thru 0x7733549 (0x4f5f bytes) 2024-09-13T04:58:52.109 INFO:teuthology.orchestra.run.smithi001.stdout:2778 read 0x7276af4 thru 0x727b8fd (0x4e0a bytes) 2024-09-13T04:58:52.109 INFO:teuthology.orchestra.run.smithi001.stdout:2780 read 0x9620269 thru 0x962d7e2 (0xd57a bytes) 2024-09-13T04:58:52.109 INFO:teuthology.orchestra.run.smithi001.stdout:2781 punch from 0x14ef936 to 0x14f3439, (0x3b03 bytes) 2024-09-13T04:58:52.109 INFO:teuthology.orchestra.run.smithi001.stdout:2782 writesame 0x7c39ed5 thru 0x7c4534d data_size 0x1(0xb479 bytes) 2024-09-13T04:58:52.111 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:52.111 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:52.124 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:52.119+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:52.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:52.161 INFO:teuthology.orchestra.run.smithi001.stdout:2783 clone 19 order 23 su 524288 sc 6 2024-09-13T04:58:52.225 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:52.219+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:52.285 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:52.285 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:52.301 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:52.342 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:52.339+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:52.426 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:52.423+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:52.469 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:52.470 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:52.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:52.499 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:58:52.499 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T04:58:52.527 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:52.523+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:52.615 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:52.628 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:52.627+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:52.657 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:52.657 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:52.675 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:52.728 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:52.727+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:52.771 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:52.771 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:52.788 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:52.788 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:52.804 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:52.804 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:52.828 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:52.823+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:52.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:52.836 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:52.843 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:52.843 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:52.843 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:52.843 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:52.857 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:52.870 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:52.886 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T04:58:52.894 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:52.909 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T04:58:52.917 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:52.917 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:52.941 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:52.941 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:52.942 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:52.935+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:53.003 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:53.003 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:53.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:53.070 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:53.067+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:53.131 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:53.127+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:53.160 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.026871882999998, 2024-09-13T04:58:53.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:58:53.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:58:53.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.024431223999997, 2024-09-13T04:58:53.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:58:53.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.735198930000003, 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:58:53.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:58:53.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:53.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:53.209 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.177599282999999, 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:53.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:58:53.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:58:53.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:58:53.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:53.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:58:53.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:58:53.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.163088231, 2024-09-13T04:58:53.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:58:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:58:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70501 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 901120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9240402369999998, 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048712395999999998, 2024-09-13T04:58:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70501 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266809+0000", 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266812+0000", 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3970000000000001e-06 2024-09-13T04:58:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266816+0000", 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.729e-06 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266820+0000", 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0690000000000003e-06 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315355+0000", 2024-09-13T04:58:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048534985000000003 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315417+0000", 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1765000000000002e-05 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315522+0000", 2024-09-13T04:58:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105142 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70502 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 905216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9240045009999998, 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048974827999999998, 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70502 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266845+0000", 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266851+0000", 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266857+0000", 2024-09-13T04:58:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7180000000000001e-06 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266861+0000", 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6569999999999996e-06 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315576+0000", 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048714396 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315687+0000", 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111118 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315821+0000", 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013375400000000001 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70503 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 860160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.231 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9239513669999999, 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049096338000000003, 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70503 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:58:53.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266898+0000", 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266901+0000", 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266906+0000", 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4600000000000002e-06 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266911+0000", 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7570000000000004e-06 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315859+0000", 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048947955000000001 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315920+0000", 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0906000000000003e-05 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315995+0000", 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5185000000000006e-05 2024-09-13T04:58:53.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70504 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 864256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9239357410000002, 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049340255, 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70504 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266917+0000", 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7180000000000001e-06 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266922+0000", 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.028e-06 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266926+0000", 2024-09-13T04:58:53.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4750000000000004e-06 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316035+0000", 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049109070999999997 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316136+0000", 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010061 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316255+0000", 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118884 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70505 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 868352~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9239204060000001, 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049486095000000001, 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70505 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:58:53.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266929+0000", 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266932+0000", 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.306e-06 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266936+0000", 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8230000000000002e-06 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266941+0000", 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.279e-06 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316291+0000", 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049349968000000001 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316344+0000", 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3659999999999998e-05 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316416+0000", 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1606999999999994e-05 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.245 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70506 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 856064~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.923905923, 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049736427, 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70506 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266946+0000", 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3240000000000001e-06 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266951+0000", 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7489999999999998e-06 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266956+0000", 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9180000000000002e-06 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316460+0000", 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049503783000000003 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316555+0000", 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316681+0000", 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125513 2024-09-13T04:58:53.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70507 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 872448~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9238757870000001, 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049868537999999997, 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.251 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70507 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266977+0000", 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8459999999999999e-06 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266982+0000", 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3179999999999997e-06 2024-09-13T04:58:53.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266986+0000", 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4320000000000001e-06 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316715+0000", 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049728886 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316772+0000", 2024-09-13T04:58:53.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6805000000000002e-05 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316843+0000", 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0928999999999998e-05 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70508 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 909312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9238485939999999, 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050098392999999998, 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70508 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267001+0000", 2024-09-13T04:58:53.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267004+0000", 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2589999999999999e-06 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267009+0000", 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4410000000000001e-06 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267014+0000", 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6319999999999999e-06 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316884+0000", 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049869956 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316982+0000", 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8472000000000002e-05 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317100+0000", 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117977 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70509 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 876544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9238331830000002, 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050242336999999998, 2024-09-13T04:58:53.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70509 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267019+0000", 2024-09-13T04:58:53.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1969999999999999e-06 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267024+0000", 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267028+0000", 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2169999999999996e-06 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317133+0000", 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050105246999999999 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317190+0000", 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6623000000000003e-05 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317259+0000", 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9599999999999998e-05 2024-09-13T04:58:53.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70510 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 913408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9238186289999999, 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050487833000000003, 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70510 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267031+0000", 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267033+0000", 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5660000000000001e-06 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267038+0000", 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7650000000000001e-06 2024-09-13T04:58:53.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267042+0000", 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.233e-06 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317301+0000", 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050258540999999997 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317403+0000", 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101727 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317519+0000", 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116685 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70511 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 884736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.269 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9238044690000002, 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050628583999999997, 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70511 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:58:53.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267045+0000", 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267048+0000", 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1430000000000001e-06 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267054+0000", 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7950000000000002e-06 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267059+0000", 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.318e-06 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317553+0000", 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050494202000000002 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317605+0000", 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2039000000000001e-05 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317674+0000", 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999996e-05 2024-09-13T04:58:53.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70512 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 880640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9237526579999997, 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050833717, 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70512 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267097+0000", 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267100+0000", 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7630000000000001e-06 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267105+0000", 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2850000000000002e-06 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267109+0000", 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5070000000000002e-06 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317716+0000", 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050606774 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317810+0000", 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3851000000000007e-05 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317931+0000", 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121177 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.278 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70513 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 892928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9237369549999999, 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050973768000000003, 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70513 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267115+0000", 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2139999999999999e-06 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267122+0000", 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9639999999999997e-06 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267127+0000", 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4789999999999998e-06 2024-09-13T04:58:53.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317967+0000", 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050840242000000001 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318019+0000", 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.189e-05 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318087+0000", 2024-09-13T04:58:53.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8312999999999995e-05 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70514 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9237198819999999, 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051225729999999997, 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.283 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70514 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267136+0000", 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1399999999999997e-06 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267141+0000", 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0690000000000001e-06 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267145+0000", 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3830000000000002e-06 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318126+0000", 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050980481000000001 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318237+0000", 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110851 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318356+0000", 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119499 2024-09-13T04:58:53.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70515 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 897024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9236678469999999, 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051324634000000001, 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:53.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70515 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:53.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267184+0000", 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6529999999999998e-06 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267190+0000", 2024-09-13T04:58:53.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267194+0000", 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4859999999999999e-06 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318381+0000", 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051186785999999998 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:53.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318436+0000", 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5180999999999999e-05 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318507+0000", 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0753999999999994e-05 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:53.292 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:53.294 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:53.294 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:53.291+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:53.394 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:53.391+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76739 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.409326+0000", 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.050157974000000001, 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050201699000000002, 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.489 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76739 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409326+0000", 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409326+0000", 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409326+0000", 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409328+0000", 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409334+0000", 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409339+0000", 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2240000000000003e-06 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.492 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76771 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.413286+0000", 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.046197698000000002, 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.046384307, 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76771 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413286+0000", 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413286+0000", 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413286+0000", 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413288+0000", 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.934e-06 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413297+0000", 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7190000000000007e-06 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413302+0000", 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9640000000000002e-06 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.495 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76803 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421297+0000", 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038186502999999997, 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038479606, 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76803 2024-09-13T04:58:53.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421297+0000", 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421297+0000", 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421297+0000", 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421299+0000", 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2919999999999998e-06 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421307+0000", 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7819999999999994e-06 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421314+0000", 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7530000000000004e-06 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76835 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425442+0000", 2024-09-13T04:58:53.499 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034041467999999998, 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034417185000000003, 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76835 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425442+0000", 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425442+0000", 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425442+0000", 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425443+0000", 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2130000000000001e-06 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425446+0000", 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9890000000000001e-06 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425450+0000", 2024-09-13T04:58:53.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4479999999999999e-06 2024-09-13T04:58:53.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.503 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76867 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433207+0000", 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026276981000000001, 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026705361, 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76867 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433207+0000", 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433207+0000", 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433207+0000", 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433208+0000", 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4139999999999999e-06 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433213+0000", 2024-09-13T04:58:53.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433216+0000", 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1159999999999999e-06 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76899 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441282+0000", 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018201599999999998, 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018731867999999999, 2024-09-13T04:58:53.507 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76899 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441282+0000", 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441282+0000", 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441282+0000", 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441283+0000", 2024-09-13T04:58:53.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3319999999999999e-06 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441287+0000", 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.016e-06 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441289+0000", 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9069999999999999e-06 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76931 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449276+0000", 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010208195, 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010840709, 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76931 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449276+0000", 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449276+0000", 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449276+0000", 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449277+0000", 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1680000000000001e-06 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449280+0000", 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.92e-06 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449282+0000", 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7989999999999998e-06 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76963 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457187+0000", 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0022964970000000002, 2024-09-13T04:58:53.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003032248, 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76963 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457187+0000", 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457187+0000", 2024-09-13T04:58:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453322+0000", 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9961348 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457191+0000", 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0038687389999999999 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457199+0000", 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3569999999999998e-06 2024-09-13T04:58:53.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457204+0000", 2024-09-13T04:58:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.369e-06 2024-09-13T04:58:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76740 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.409343+0000", 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.050141009, 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05097492, 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76740 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.519 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409343+0000", 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409343+0000", 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409343+0000", 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409345+0000", 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3690000000000001e-06 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409351+0000", 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9209999999999997e-06 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409357+0000", 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1829999999999999e-06 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76772 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.413307+0000", 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.046177231999999999, 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.047065773999999998, 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76772 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413307+0000", 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413307+0000", 2024-09-13T04:58:53.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413306+0000", 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413309+0000", 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1950000000000002e-06 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413314+0000", 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8089999999999998e-06 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413320+0000", 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4609999999999999e-06 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.526 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76804 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421318+0000", 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038165643999999999, 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039140637999999998, 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76804 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421318+0000", 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421318+0000", 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421318+0000", 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421326+0000", 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8110000000000003e-06 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421332+0000", 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3099999999999997e-06 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421338+0000", 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2280000000000003e-06 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.529 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76836 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425453+0000", 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034031035000000001, 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035108601000000003, 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.530 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76836 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425453+0000", 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425453+0000", 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425453+0000", 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425454+0000", 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2529999999999999e-06 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425457+0000", 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0149999999999999e-06 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425460+0000", 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7149999999999998e-06 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76868 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.533 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433219+0000", 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026265398999999998, 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.02744678, 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76868 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433219+0000", 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433219+0000", 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433218+0000", 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433220+0000", 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.279e-06 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433223+0000", 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.033e-06 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433225+0000", 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.632e-06 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76900 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441292+0000", 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018191777999999999, 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019475517000000001, 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76900 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441292+0000", 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441292+0000", 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441292+0000", 2024-09-13T04:58:53.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441293+0000", 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.333e-06 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441296+0000", 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0060000000000001e-06 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441299+0000", 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6410000000000002e-06 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76932 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449285+0000", 2024-09-13T04:58:53.540 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010199366, 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011541106000000001, 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76932 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449285+0000", 2024-09-13T04:58:53.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449285+0000", 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449284+0000", 2024-09-13T04:58:53.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449286+0000", 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.316e-06 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449289+0000", 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9490000000000001e-06 2024-09-13T04:58:53.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449291+0000", 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7520000000000001e-06 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.544 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76964 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457208+0000", 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.002275558, 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036903159999999999, 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76964 2024-09-13T04:58:53.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457208+0000", 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457208+0000", 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457208+0000", 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457210+0000", 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6700000000000001e-06 2024-09-13T04:58:53.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457214+0000", 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2869999999999998e-06 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457217+0000", 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.74e-06 2024-09-13T04:58:53.548 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76741 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.409363+0000", 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.050120840999999999, 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.051637857000000002, 2024-09-13T04:58:53.549 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76741 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409363+0000", 2024-09-13T04:58:53.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409363+0000", 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409363+0000", 2024-09-13T04:58:53.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409365+0000", 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3580000000000001e-06 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409372+0000", 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.379e-06 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409381+0000", 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6349999999999995e-06 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76773 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.413325+0000", 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.046159182999999999, 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.047796668, 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.553 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76773 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413325+0000", 2024-09-13T04:58:53.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413325+0000", 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413324+0000", 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413327+0000", 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.142e-06 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413335+0000", 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0930000000000003e-06 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413340+0000", 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1449999999999999e-06 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.556 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76805 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421344+0000", 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038139885999999998, 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039885939000000002, 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76805 2024-09-13T04:58:53.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421344+0000", 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421344+0000", 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421344+0000", 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421346+0000", 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3819999999999998e-06 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421364+0000", 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7785e-05 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421372+0000", 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3450000000000006e-06 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76837 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429183+0000", 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030300508, 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.032110172999999999, 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.560 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76837 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429183+0000", 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429183+0000", 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425462+0000", 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9962788 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429187+0000", 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037254779999999999 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429200+0000", 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2293000000000001e-05 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429206+0000", 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8760000000000002e-06 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76869 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433228+0000", 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026256175, 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.028117303, 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76869 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433228+0000", 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433228+0000", 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433228+0000", 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433229+0000", 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.279e-06 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433232+0000", 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5619999999999999e-06 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433235+0000", 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.661e-06 2024-09-13T04:58:53.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76901 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441301+0000", 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018182856000000001, 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020101510999999999, 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76901 2024-09-13T04:58:53.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441301+0000", 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441301+0000", 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441301+0000", 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441306+0000", 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6129999999999997e-06 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441309+0000", 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1820000000000002e-06 2024-09-13T04:58:53.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441311+0000", 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7070000000000001e-06 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76933 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449294+0000", 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0101904, 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012162068999999999, 2024-09-13T04:58:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76933 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449294+0000", 2024-09-13T04:58:53.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449294+0000", 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449293+0000", 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449295+0000", 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.274e-06 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449298+0000", 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0460000000000001e-06 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449300+0000", 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5830000000000001e-06 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76965 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457220+0000", 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0022635250000000002, 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0043229979999999998, 2024-09-13T04:58:53.574 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76965 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457220+0000", 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457220+0000", 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457220+0000", 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457222+0000", 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.491e-06 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457226+0000", 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0330000000000002e-06 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457228+0000", 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7039999999999999e-06 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76742 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.409387+0000", 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.050096860999999999, 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052258855999999999, 2024-09-13T04:58:53.578 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76742 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409387+0000", 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409387+0000", 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409387+0000", 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409389+0000", 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3949999999999999e-06 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409395+0000", 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2879999999999998e-06 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409400+0000", 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7269999999999998e-06 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76774 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417173+0000", 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042310849999999997, 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.044577269000000003, 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76774 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417173+0000", 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417173+0000", 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.413344+0000", 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.996171 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417178+0000", 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0038339340000000002 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417187+0000", 2024-09-13T04:58:53.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4970000000000007e-06 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417194+0000", 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8639999999999998e-06 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76806 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421378+0000", 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038106159000000001, 2024-09-13T04:58:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040474336999999999, 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76806 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421378+0000", 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421378+0000", 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421377+0000", 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421380+0000", 2024-09-13T04:58:53.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4389999999999999e-06 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421386+0000", 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1430000000000003e-06 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421391+0000", 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7450000000000003e-06 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76838 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429230+0000", 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030254303, 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.032678008000000001, 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76838 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429230+0000", 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429230+0000", 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429229+0000", 2024-09-13T04:58:53.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429232+0000", 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0929999999999999e-06 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429245+0000", 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2425999999999999e-05 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429251+0000", 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76870 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433238+0000", 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026246446, 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.028721747999999998, 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76870 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433238+0000", 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433238+0000", 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433237+0000", 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433239+0000", 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.319e-06 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433242+0000", 2024-09-13T04:58:53.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.937e-06 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433244+0000", 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6220000000000001e-06 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76902 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441314+0000", 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018170131999999999, 2024-09-13T04:58:53.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020755358000000002, 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76902 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441314+0000", 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441314+0000", 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441314+0000", 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441315+0000", 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.266e-06 2024-09-13T04:58:53.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441318+0000", 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9359999999999999e-06 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441321+0000", 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2389999999999998e-06 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.599 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76934 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449302+0000", 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010181519999999999, 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012869281, 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76934 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449302+0000", 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449302+0000", 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449302+0000", 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449304+0000", 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3179999999999999e-06 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449307+0000", 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0680000000000001e-06 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449309+0000", 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5670000000000002e-06 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76966 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457231+0000", 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0022531500000000002, 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0050351090000000003, 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76966 2024-09-13T04:58:53.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457231+0000", 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457231+0000", 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457231+0000", 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457232+0000", 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2810000000000001e-06 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457235+0000", 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.994e-06 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457237+0000", 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5289999999999998e-06 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.606 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76743 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.409405+0000", 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.050079431000000001, 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052913227, 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76743 2024-09-13T04:58:53.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409405+0000", 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409405+0000", 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409404+0000", 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409413+0000", 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5019999999999992e-06 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409420+0000", 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4460000000000004e-06 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409423+0000", 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1659999999999998e-06 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.610 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76775 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417209+0000", 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042274620999999998, 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.045208287, 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76775 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.611 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417209+0000", 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417209+0000", 2024-09-13T04:58:53.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417209+0000", 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417212+0000", 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0680000000000001e-06 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417221+0000", 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3780000000000007e-06 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417229+0000", 2024-09-13T04:58:53.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7030000000000006e-06 2024-09-13T04:58:53.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.615 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.615 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76807 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.615 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421407+0000", 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038077090000000001, 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041110834999999998, 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76807 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421407+0000", 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421407+0000", 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421407+0000", 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421408+0000", 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6199999999999999e-06 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421412+0000", 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7120000000000002e-06 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421415+0000", 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0019999999999998e-06 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76839 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429257+0000", 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030227460000000001, 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.033431735999999997, 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76839 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429257+0000", 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429257+0000", 2024-09-13T04:58:53.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429256+0000", 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429259+0000", 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.452e-06 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429267+0000", 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5229999999999995e-06 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429272+0000", 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1699999999999996e-06 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76871 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433246+0000", 2024-09-13T04:58:53.622 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026237574, 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029532593999999999, 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76871 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433246+0000", 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433246+0000", 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433246+0000", 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433247+0000", 2024-09-13T04:58:53.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3009999999999999e-06 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433254+0000", 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3729999999999998e-06 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433257+0000", 2024-09-13T04:58:53.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1140000000000002e-06 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76903 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441324+0000", 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018160401, 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021512129000000001, 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76903 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441324+0000", 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441324+0000", 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441323+0000", 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441325+0000", 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2559999999999999e-06 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441328+0000", 2024-09-13T04:58:53.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9110000000000002e-06 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441330+0000", 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6419999999999999e-06 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76935 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449311+0000", 2024-09-13T04:58:53.629 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010172507000000001, 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013622822999999999, 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76935 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449311+0000", 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449311+0000", 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449311+0000", 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449313+0000", 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.381e-06 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449317+0000", 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1590000000000003e-06 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449320+0000", 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7420000000000002e-06 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.632 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76967 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457240+0000", 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0022444399999999999, 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0057996999999999996, 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.633 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76967 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457240+0000", 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457240+0000", 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457239+0000", 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457241+0000", 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.393e-06 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457244+0000", 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9529999999999999e-06 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457246+0000", 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4660000000000002e-06 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76744 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.409427+0000", 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.050056574, 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053715007000000002, 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.636 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76744 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409427+0000", 2024-09-13T04:58:53.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409427+0000", 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409427+0000", 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409429+0000", 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5719999999999999e-06 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409433+0000", 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4399999999999998e-06 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.409436+0000", 2024-09-13T04:58:53.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.897e-06 2024-09-13T04:58:53.640 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.640 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.640 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76776 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417234+0000", 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042249918999999997, 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.046009682000000003, 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76776 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417234+0000", 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417234+0000", 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417233+0000", 2024-09-13T04:58:53.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417236+0000", 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.632e-06 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417248+0000", 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1793999999999999e-05 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417253+0000", 2024-09-13T04:58:53.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1580000000000004e-06 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76808 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421418+0000", 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038065888999999999, 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041882513000000003, 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.644 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76808 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421418+0000", 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421418+0000", 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421418+0000", 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421419+0000", 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.378e-06 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421422+0000", 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0060000000000001e-06 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421425+0000", 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.706e-06 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76840 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429277+0000", 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030207320999999999, 2024-09-13T04:58:53.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034101823000000003, 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76840 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429277+0000", 2024-09-13T04:58:53.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429277+0000", 2024-09-13T04:58:53.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429276+0000", 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429293+0000", 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6985000000000001e-05 2024-09-13T04:58:53.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429300+0000", 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1749999999999999e-06 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429306+0000", 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3009999999999997e-06 2024-09-13T04:58:53.652 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76872 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433259+0000", 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026224620000000001, 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030225782, 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.653 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76872 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433259+0000", 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433259+0000", 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433259+0000", 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433261+0000", 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4890000000000001e-06 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433264+0000", 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.005e-06 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433266+0000", 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7089999999999998e-06 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76904 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441333+0000", 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018151440000000001, 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.022254060999999999, 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.656 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76904 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441333+0000", 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441333+0000", 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441332+0000", 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441334+0000", 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.24e-06 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441336+0000", 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9390000000000002e-06 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441339+0000", 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6759999999999999e-06 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76936 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.659 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449322+0000", 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010162244000000001, 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014370805, 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76936 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449322+0000", 2024-09-13T04:58:53.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449322+0000", 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449322+0000", 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449323+0000", 2024-09-13T04:58:53.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.358e-06 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449326+0000", 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0340000000000001e-06 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449329+0000", 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6299999999999998e-06 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.662 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76968 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457248+0000", 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0022357869999999999, 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065455419999999997, 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76968 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457248+0000", 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457248+0000", 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457248+0000", 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457249+0000", 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.299e-06 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457254+0000", 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3970000000000004e-06 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457256+0000", 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6010000000000002e-06 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76777 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417258+0000", 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042226236, 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.046645824000000002, 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.666 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76777 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417258+0000", 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417258+0000", 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417257+0000", 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417260+0000", 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3489999999999999e-06 2024-09-13T04:58:53.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417271+0000", 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1355000000000001e-05 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417276+0000", 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7520000000000004e-06 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.669 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76809 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421428+0000", 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038056373999999997, 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042579935999999999, 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76809 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421428+0000", 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421428+0000", 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421427+0000", 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421429+0000", 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.491e-06 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421432+0000", 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0690000000000002e-06 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421435+0000", 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7209999999999999e-06 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.672 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76841 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429311+0000", 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030173285000000001, 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034776357000000001, 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.673 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76841 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429311+0000", 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429311+0000", 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429310+0000", 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429313+0000", 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3369999999999998e-06 2024-09-13T04:58:53.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429318+0000", 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5670000000000002e-06 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429323+0000", 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6589999999999998e-06 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76873 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437207+0000", 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022277350000000001, 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026947321999999999, 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.677 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76873 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437207+0000", 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437207+0000", 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433268+0000", 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9960618 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437210+0000", 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039413019999999998 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437215+0000", 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6500000000000001e-06 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437220+0000", 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1309999999999997e-06 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76905 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441341+0000", 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018142603, 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.022919762999999999, 2024-09-13T04:58:53.680 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76905 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441341+0000", 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441341+0000", 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441341+0000", 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441342+0000", 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.218e-06 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441345+0000", 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8839999999999998e-06 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441348+0000", 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6560000000000001e-06 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76937 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.683 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449331+0000", 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.01015337, 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015031858, 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76937 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449331+0000", 2024-09-13T04:58:53.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449331+0000", 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449330+0000", 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449332+0000", 2024-09-13T04:58:53.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3430000000000001e-06 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449335+0000", 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9009999999999998e-06 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449337+0000", 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.548e-06 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76969 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457259+0000", 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.002225388, 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.007205174, 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76969 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457259+0000", 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457259+0000", 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457258+0000", 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457259+0000", 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1209999999999999e-06 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457262+0000", 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9490000000000001e-06 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457265+0000", 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5339999999999998e-06 2024-09-13T04:58:53.690 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76778 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417280+0000", 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042203464000000003, 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.047264632000000001, 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.691 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76778 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417280+0000", 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417280+0000", 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417280+0000", 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417282+0000", 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.367e-06 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417289+0000", 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.127e-06 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417293+0000", 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.651e-06 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76810 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.695 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421437+0000", 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038047153, 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043168309000000002, 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76810 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421437+0000", 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421437+0000", 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421437+0000", 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421438+0000", 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.449e-06 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421441+0000", 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.994e-06 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421444+0000", 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7659999999999999e-06 2024-09-13T04:58:53.698 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76842 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429327+0000", 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030156894, 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035385255999999997, 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.699 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76842 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429327+0000", 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429327+0000", 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429327+0000", 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429329+0000", 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1730000000000002e-06 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429335+0000", 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5719999999999999e-06 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429340+0000", 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5349999999999998e-06 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76874 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437245+0000", 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022238569999999999, 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027572703, 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.702 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76874 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437245+0000", 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437245+0000", 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437245+0000", 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437247+0000", 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.731e-06 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437250+0000", 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.625e-06 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437253+0000", 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.858e-06 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.705 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76906 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441350+0000", 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018133882, 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.023574243000000002, 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76906 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441350+0000", 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441350+0000", 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441350+0000", 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441351+0000", 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.097e-06 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441356+0000", 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6240000000000001e-06 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441358+0000", 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6000000000000001e-06 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76938 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449339+0000", 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.01014459, 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015665037, 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.709 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76938 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449339+0000", 2024-09-13T04:58:53.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449339+0000", 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449339+0000", 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449344+0000", 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4900000000000002e-06 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449347+0000", 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0259999999999999e-06 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449349+0000", 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7140000000000002e-06 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76970 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457267+0000", 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.002216796, 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0078119119999999998, 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.713 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76970 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457267+0000", 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457267+0000", 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457267+0000", 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457268+0000", 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1060000000000001e-06 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457271+0000", 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9059999999999998e-06 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457274+0000", 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.716 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76779 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417298+0000", 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042186398, 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.047872789999999998, 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76779 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417298+0000", 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417298+0000", 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417297+0000", 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417300+0000", 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4949999999999998e-06 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417305+0000", 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6749999999999999e-06 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417313+0000", 2024-09-13T04:58:53.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1219999999999997e-06 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76811 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421446+0000", 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038038102999999997, 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043833768000000002, 2024-09-13T04:58:53.720 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76811 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421446+0000", 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421446+0000", 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421446+0000", 2024-09-13T04:58:53.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421447+0000", 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4610000000000001e-06 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421450+0000", 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.993e-06 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421453+0000", 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9790000000000002e-06 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76843 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429344+0000", 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030140031000000001, 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.036038406000000002, 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76843 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429344+0000", 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429344+0000", 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429344+0000", 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429346+0000", 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3e-06 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429355+0000", 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4809999999999995e-06 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429361+0000", 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1340000000000003e-06 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76875 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437256+0000", 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022228176999999998, 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.028228795000000001, 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.728 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76875 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437256+0000", 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437256+0000", 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437256+0000", 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437257+0000", 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5319999999999999e-06 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437260+0000", 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9639999999999999e-06 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437263+0000", 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6359999999999998e-06 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76907 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.731 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445040+0000", 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014443612999999999, 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020563552999999998, 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76907 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445040+0000", 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445040+0000", 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441360+0000", 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9963202 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445043+0000", 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036823749999999999 2024-09-13T04:58:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445048+0000", 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2329999999999997e-06 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445052+0000", 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.518e-06 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76939 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449352+0000", 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010132173, 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016357977999999999, 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76939 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449352+0000", 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449352+0000", 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449352+0000", 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449353+0000", 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4100000000000001e-06 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449356+0000", 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9569999999999998e-06 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449359+0000", 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6680000000000001e-06 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.737 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76971 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457277+0000", 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0022073029999999999, 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0085386699999999999, 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76971 2024-09-13T04:58:53.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457277+0000", 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457277+0000", 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457276+0000", 2024-09-13T04:58:53.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457278+0000", 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4419999999999999e-06 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457286+0000", 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9699999999999999e-06 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457289+0000", 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.399e-06 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76780 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417317+0000", 2024-09-13T04:58:53.741 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042166650999999999, 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.048601362000000002, 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76780 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417317+0000", 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417317+0000", 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417317+0000", 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417334+0000", 2024-09-13T04:58:53.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7173999999999999e-05 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417341+0000", 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0999999999999998e-06 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417347+0000", 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.288e-06 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.744 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76812 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421455+0000", 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038028646999999999, 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.044711366000000002, 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76812 2024-09-13T04:58:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421455+0000", 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421455+0000", 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421455+0000", 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421456+0000", 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2979999999999999e-06 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421459+0000", 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.982e-06 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421462+0000", 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.599e-06 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.748 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76844 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429365+0000", 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030119257, 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.036904810000000003, 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.749 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76844 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429365+0000", 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429365+0000", 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429364+0000", 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429367+0000", 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3209999999999999e-06 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429374+0000", 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3980000000000002e-06 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429380+0000", 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5999999999999997e-06 2024-09-13T04:58:53.752 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76876 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437265+0000", 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022218965, 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029113693, 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.753 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76876 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437265+0000", 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437265+0000", 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437265+0000", 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437266+0000", 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.502e-06 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437271+0000", 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.322e-06 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437274+0000", 2024-09-13T04:58:53.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.667e-06 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76908 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445066+0000", 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014418301999999999, 2024-09-13T04:58:53.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021415962, 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76908 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445066+0000", 2024-09-13T04:58:53.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445066+0000", 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445065+0000", 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445067+0000", 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.779e-06 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445072+0000", 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7180000000000004e-06 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445075+0000", 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8760000000000001e-06 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.760 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76940 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449361+0000", 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010123066, 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017227498000000001, 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76940 2024-09-13T04:58:53.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449361+0000", 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449361+0000", 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449361+0000", 2024-09-13T04:58:53.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449362+0000", 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.274e-06 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449365+0000", 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9040000000000001e-06 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449367+0000", 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5600000000000001e-06 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76972 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457292+0000", 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00219229, 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.009401029, 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76972 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457292+0000", 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457292+0000", 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457291+0000", 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457293+0000", 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3990000000000001e-06 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457296+0000", 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.174e-06 2024-09-13T04:58:53.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457299+0000", 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3510000000000002e-06 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76781 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417351+0000", 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042132854999999997, 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.049451564000000003, 2024-09-13T04:58:53.767 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76781 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417351+0000", 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417351+0000", 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417351+0000", 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417353+0000", 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4090000000000001e-06 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417371+0000", 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7499e-05 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417378+0000", 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5830000000000001e-06 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76813 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.770 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421465+0000", 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038018969, 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.045440437, 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76813 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421465+0000", 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421465+0000", 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421465+0000", 2024-09-13T04:58:53.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421466+0000", 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.328e-06 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421470+0000", 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1819999999999999e-06 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421473+0000", 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7870000000000002e-06 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76845 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429384+0000", 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030100192000000001, 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037623950000000003, 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76845 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429384+0000", 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429384+0000", 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429384+0000", 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429385+0000", 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6559999999999999e-06 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429390+0000", 2024-09-13T04:58:53.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.605e-06 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429393+0000", 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7829999999999999e-06 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76877 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437276+0000", 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022207574000000001, 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029839982000000001, 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76877 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437276+0000", 2024-09-13T04:58:53.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437276+0000", 2024-09-13T04:58:53.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437276+0000", 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437287+0000", 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1099e-05 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437292+0000", 2024-09-13T04:58:53.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9799999999999998e-06 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437295+0000", 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.118e-06 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76909 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445085+0000", 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014399248, 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.022133685, 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76909 2024-09-13T04:58:53.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445085+0000", 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445085+0000", 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445084+0000", 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445086+0000", 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5e-06 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445091+0000", 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9869999999999999e-06 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445094+0000", 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.898e-06 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76941 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449392+0000", 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010091904, 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017937205000000001, 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.790 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76941 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449392+0000", 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449392+0000", 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449392+0000", 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449393+0000", 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6479999999999999e-06 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449402+0000", 2024-09-13T04:58:53.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8400000000000001e-06 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449405+0000", 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9249999999999999e-06 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76973 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457302+0000", 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0021822870000000002, 2024-09-13T04:58:53.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010129783, 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76973 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457302+0000", 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457302+0000", 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457301+0000", 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457303+0000", 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2279999999999999e-06 2024-09-13T04:58:53.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457306+0000", 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9790000000000002e-06 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457308+0000", 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.57e-06 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76782 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417385+0000", 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042098991000000002, 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050155564999999999, 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76782 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417385+0000", 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417385+0000", 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417385+0000", 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417387+0000", 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.587e-06 2024-09-13T04:58:53.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417396+0000", 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4400000000000005e-06 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417401+0000", 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:58:53.800 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76814 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421475+0000", 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038008477999999998, 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.046167022000000002, 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.801 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76814 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421475+0000", 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421475+0000", 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421475+0000", 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421477+0000", 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3260000000000001e-06 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421480+0000", 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0189999999999998e-06 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421482+0000", 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7520000000000001e-06 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76846 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429395+0000", 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030089036999999999, 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038355001999999999, 2024-09-13T04:58:53.804 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76846 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429395+0000", 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429395+0000", 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429395+0000", 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429396+0000", 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.393e-06 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429400+0000", 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9899999999999999e-06 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429403+0000", 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6089999999999999e-06 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76878 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437298+0000", 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022186104000000002, 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030554558999999998, 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76878 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437298+0000", 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437298+0000", 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437298+0000", 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437299+0000", 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.331e-06 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437302+0000", 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.123e-06 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437305+0000", 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.349e-06 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76910 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445096+0000", 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014387726, 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.022861804999999999, 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.811 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76910 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445096+0000", 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445096+0000", 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445096+0000", 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445097+0000", 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.429e-06 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445105+0000", 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1409999999999999e-06 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445108+0000", 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.095e-06 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76942 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449409+0000", 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010075316000000001, 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018650942, 2024-09-13T04:58:53.814 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76942 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449409+0000", 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449409+0000", 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449408+0000", 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449410+0000", 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.375e-06 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449413+0000", 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1640000000000001e-06 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449416+0000", 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6589999999999999e-06 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.817 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76974 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457311+0000", 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0021733630000000002, 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010852437, 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76974 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457311+0000", 2024-09-13T04:58:53.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457311+0000", 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457310+0000", 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457315+0000", 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7060000000000003e-06 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457319+0000", 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2490000000000003e-06 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457322+0000", 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5639999999999999e-06 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76783 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417405+0000", 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042078931, 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050857483000000002, 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.821 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76783 2024-09-13T04:58:53.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.822 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417405+0000", 2024-09-13T04:58:53.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417405+0000", 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417405+0000", 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417407+0000", 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5320000000000001e-06 2024-09-13T04:58:53.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417416+0000", 2024-09-13T04:58:53.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9039999999999998e-06 2024-09-13T04:58:53.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417423+0000", 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0010000000000004e-06 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76815 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425102+0000", 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034381981999999998, 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043212643000000002, 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.826 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76815 2024-09-13T04:58:53.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.826 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425102+0000", 2024-09-13T04:58:53.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425102+0000", 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421484+0000", 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9963822 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425107+0000", 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036229729999999998 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425116+0000", 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7250000000000003e-06 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425124+0000", 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3639999999999999e-06 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76847 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429405+0000", 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030078892999999999, 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038973720000000003, 2024-09-13T04:58:53.829 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76847 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429405+0000", 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429405+0000", 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429405+0000", 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429406+0000", 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3e-06 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429410+0000", 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9609999999999999e-06 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429413+0000", 2024-09-13T04:58:53.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6460000000000002e-06 2024-09-13T04:58:53.833 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.833 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.833 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76879 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437308+0000", 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022176353999999999, 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031144228999999999, 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76879 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.834 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437308+0000", 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437308+0000", 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437307+0000", 2024-09-13T04:58:53.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437309+0000", 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.663e-06 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437312+0000", 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.971e-06 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437315+0000", 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7020000000000001e-06 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76911 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445111+0000", 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014373423999999999, 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.023446291000000001, 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76911 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445111+0000", 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445111+0000", 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445110+0000", 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445112+0000", 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5939999999999999e-06 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445115+0000", 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2729999999999998e-06 2024-09-13T04:58:53.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445118+0000", 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5069999999999999e-06 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76943 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449418+0000", 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010066031, 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019241114, 2024-09-13T04:58:53.840 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76943 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449418+0000", 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449418+0000", 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449418+0000", 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449419+0000", 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.466e-06 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449422+0000", 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9359999999999999e-06 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449425+0000", 2024-09-13T04:58:53.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2399999999999999e-06 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76975 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457324+0000", 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0021594639999999998, 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011441059999999999, 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76975 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457324+0000", 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457324+0000", 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457324+0000", 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457326+0000", 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2699999999999999e-06 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457331+0000", 2024-09-13T04:58:53.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3260000000000003e-06 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457333+0000", 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76784 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417428+0000", 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042056410000000002, 2024-09-13T04:58:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.051442288000000003, 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76784 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417428+0000", 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417428+0000", 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417427+0000", 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417430+0000", 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5840000000000002e-06 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417436+0000", 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0179999999999998e-06 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417443+0000", 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9e-06 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.850 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76816 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425138+0000", 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034346433000000003, 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043841142999999999, 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76816 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425138+0000", 2024-09-13T04:58:53.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425138+0000", 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425137+0000", 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425145+0000", 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3089999999999996e-06 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425152+0000", 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.46e-06 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425158+0000", 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.635e-06 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.853 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76848 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429415+0000", 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030068945, 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039658864000000002, 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76848 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429415+0000", 2024-09-13T04:58:53.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429415+0000", 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429415+0000", 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429416+0000", 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3209999999999999e-06 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429420+0000", 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.912e-06 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429423+0000", 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.644e-06 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.857 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76880 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437317+0000", 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022167151999999999, 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031827271999999997, 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.858 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76880 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437317+0000", 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437317+0000", 2024-09-13T04:58:53.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437317+0000", 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437318+0000", 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.356e-06 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437321+0000", 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9900000000000002e-06 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437325+0000", 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8589999999999998e-06 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.861 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76912 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445120+0000", 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014364227, 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024123136999999999, 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.862 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76912 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445120+0000", 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445120+0000", 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445120+0000", 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445121+0000", 2024-09-13T04:58:53.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2929999999999999e-06 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445124+0000", 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9500000000000001e-06 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445127+0000", 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3790000000000002e-06 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76944 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449427+0000", 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010056481000000001, 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019920386000000002, 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76944 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449427+0000", 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449427+0000", 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449427+0000", 2024-09-13T04:58:53.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449429+0000", 2024-09-13T04:58:53.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.282e-06 2024-09-13T04:58:53.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449431+0000", 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9450000000000002e-06 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449434+0000", 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7439999999999999e-06 2024-09-13T04:58:53.869 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76976 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457336+0000", 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0021481059999999999, 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012133378, 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.870 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76976 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457336+0000", 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457336+0000", 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457336+0000", 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457337+0000", 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3260000000000001e-06 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457342+0000", 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1580000000000004e-06 2024-09-13T04:58:53.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457345+0000", 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5579999999999999e-06 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76785 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417468+0000", 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042015481, 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052104589, 2024-09-13T04:58:53.873 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76785 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417468+0000", 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417468+0000", 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417468+0000", 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417470+0000", 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7260000000000001e-06 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417474+0000", 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9559999999999999e-06 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417477+0000", 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0010000000000002e-06 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.876 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76817 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425164+0000", 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034320449000000003, 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.044517046999999997, 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76817 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425164+0000", 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425164+0000", 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425163+0000", 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425166+0000", 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5229999999999998e-06 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425175+0000", 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5789999999999993e-06 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425181+0000", 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7100000000000004e-06 2024-09-13T04:58:53.879 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76849 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429425+0000", 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030059046999999998, 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.040357584000000002, 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.880 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76849 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429425+0000", 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429425+0000", 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429425+0000", 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429426+0000", 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3e-06 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429430+0000", 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.67e-06 2024-09-13T04:58:53.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429433+0000", 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7400000000000002e-06 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76881 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437328+0000", 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022156209, 2024-09-13T04:58:53.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.032561642000000002, 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76881 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437328+0000", 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437328+0000", 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437328+0000", 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437329+0000", 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.277e-06 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437332+0000", 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0060000000000001e-06 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437335+0000", 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.836e-06 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.886 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76913 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445129+0000", 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014354619000000001, 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024861246, 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.887 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76913 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445129+0000", 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445129+0000", 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445129+0000", 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445134+0000", 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9280000000000001e-06 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445141+0000", 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2910000000000002e-06 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445144+0000", 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0429999999999999e-06 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76945 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453064+0000", 2024-09-13T04:58:53.891 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.006419714, 2024-09-13T04:58:53.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017032160000000001, 2024-09-13T04:58:53.891 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.891 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76945 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453064+0000", 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453064+0000", 2024-09-13T04:58:53.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449436+0000", 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9963717 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453067+0000", 2024-09-13T04:58:53.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0036313420000000001 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453075+0000", 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3390000000000004e-06 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453079+0000", 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.777e-06 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.894 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76977 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.457347+0000", 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0021367729999999998, 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012851071, 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.895 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76977 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457347+0000", 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457347+0000", 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457347+0000", 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457348+0000", 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.269e-06 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457351+0000", 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0240000000000002e-06 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457354+0000", 2024-09-13T04:58:53.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6900000000000001e-06 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76786 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417479+0000", 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.042004558999999997, 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052826132999999997, 2024-09-13T04:58:53.899 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76786 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417479+0000", 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417479+0000", 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417479+0000", 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417481+0000", 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.393e-06 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417484+0000", 2024-09-13T04:58:53.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0230000000000001e-06 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417486+0000", 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.576e-06 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76818 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.903 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425185+0000", 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034298544, 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.045220560999999999, 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76818 2024-09-13T04:58:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425185+0000", 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425185+0000", 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425185+0000", 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425188+0000", 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5160000000000001e-06 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425194+0000", 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0340000000000002e-06 2024-09-13T04:58:53.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425198+0000", 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8180000000000003e-06 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76850 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429437+0000", 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030046643000000001, 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041075229999999997, 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.907 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76850 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429437+0000", 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429437+0000", 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429437+0000", 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429439+0000", 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4339999999999999e-06 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429447+0000", 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3130000000000001e-06 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429452+0000", 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.955e-06 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.910 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76882 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437337+0000", 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022147159, 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.033283313000000002, 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76882 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.911 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437337+0000", 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437337+0000", 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437337+0000", 2024-09-13T04:58:53.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437338+0000", 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3960000000000001e-06 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437341+0000", 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9739999999999998e-06 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437344+0000", 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7439999999999999e-06 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76914 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445147+0000", 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014336887, 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025573357000000001, 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.914 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76914 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445147+0000", 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445147+0000", 2024-09-13T04:58:53.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445147+0000", 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445148+0000", 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4339999999999999e-06 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445152+0000", 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1930000000000003e-06 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445155+0000", 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6630000000000002e-06 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.917 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76946 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453102+0000", 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.006382197, 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017724783000000001, 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76946 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453102+0000", 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453102+0000", 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453101+0000", 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453103+0000", 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6929999999999999e-06 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453107+0000", 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453110+0000", 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8720000000000002e-06 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.920 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76978 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461211+0000", 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9982729, 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0097186149999999999, 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76978 2024-09-13T04:58:53.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461211+0000", 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461211+0000", 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.457356+0000", 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9961452 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461214+0000", 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003857924 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461220+0000", 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4230000000000002e-06 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461224+0000", 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6559999999999998e-06 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76787 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417489+0000", 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.041994663000000002, 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053549548000000002, 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.924 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.925 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.925 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.925 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.925 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76787 2024-09-13T04:58:53.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.925 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417489+0000", 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417489+0000", 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417489+0000", 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417490+0000", 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4780000000000001e-06 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417493+0000", 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.056e-06 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417496+0000", 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5969999999999999e-06 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76819 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425203+0000", 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034281317999999998, 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.045929352999999999, 2024-09-13T04:58:53.928 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76819 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425203+0000", 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425203+0000", 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425202+0000", 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425204+0000", 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3640000000000002e-06 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425210+0000", 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7270000000000004e-06 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425215+0000", 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.707e-06 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76851 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429455+0000", 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.03002879, 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.041732664000000003, 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.932 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76851 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429455+0000", 2024-09-13T04:58:53.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429455+0000", 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429455+0000", 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429457+0000", 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.669e-06 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429461+0000", 2024-09-13T04:58:53.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2649999999999998e-06 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429464+0000", 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4300000000000002e-06 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76883 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437346+0000", 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022137904999999999, 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.033894725000000001, 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76883 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437346+0000", 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437346+0000", 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437346+0000", 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437347+0000", 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3260000000000001e-06 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437350+0000", 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.942e-06 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437353+0000", 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7240000000000001e-06 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76915 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445157+0000", 2024-09-13T04:58:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014326563, 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026136039999999999, 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76915 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445157+0000", 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445157+0000", 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445157+0000", 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445159+0000", 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.364e-06 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445162+0000", 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9869999999999999e-06 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445165+0000", 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3579999999999999e-06 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76947 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453114+0000", 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.006370438, 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018229111999999999, 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.944 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76947 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453114+0000", 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453114+0000", 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453113+0000", 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453115+0000", 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4589999999999999e-06 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453118+0000", 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.988e-06 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453120+0000", 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6110000000000001e-06 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76979 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.947 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461258+0000", 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9982262, 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010151907999999999, 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76979 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461258+0000", 2024-09-13T04:58:53.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461258+0000", 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461257+0000", 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461259+0000", 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.765e-06 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461263+0000", 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.569e-06 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461266+0000", 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7389999999999999e-06 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76788 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417498+0000", 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.041985479999999999, 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054013654000000001, 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76788 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417498+0000", 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417498+0000", 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417498+0000", 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417500+0000", 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.339e-06 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417503+0000", 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9660000000000001e-06 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417505+0000", 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5890000000000001e-06 2024-09-13T04:58:53.953 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76820 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425219+0000", 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034264920999999997, 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.046396283000000003, 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76820 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425219+0000", 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425219+0000", 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425219+0000", 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425221+0000", 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425227+0000", 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7409999999999998e-06 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425231+0000", 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8849999999999998e-06 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76852 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429467+0000", 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.030017294, 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042256833000000001, 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.957 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76852 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429467+0000", 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429467+0000", 2024-09-13T04:58:53.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429466+0000", 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429468+0000", 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.406e-06 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429472+0000", 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.228e-06 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429475+0000", 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3340000000000002e-06 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76884 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.960 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437355+0000", 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022128968999999998, 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034431680999999999, 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76884 2024-09-13T04:58:53.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437355+0000", 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437355+0000", 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437355+0000", 2024-09-13T04:58:53.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437356+0000", 2024-09-13T04:58:53.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.316e-06 2024-09-13T04:58:53.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437360+0000", 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1819999999999999e-06 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437363+0000", 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7549999999999999e-06 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.964 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76916 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445167+0000", 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014316870000000001, 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026673767000000001, 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76916 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445167+0000", 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445167+0000", 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445167+0000", 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445168+0000", 2024-09-13T04:58:53.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.235e-06 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445171+0000", 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9289999999999998e-06 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445174+0000", 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6699999999999998e-06 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.968 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76948 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453123+0000", 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.006361322, 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018768211999999999, 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.969 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76948 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453123+0000", 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453123+0000", 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453122+0000", 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453124+0000", 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4810000000000001e-06 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453127+0000", 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.943e-06 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453129+0000", 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6029999999999999e-06 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76980 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461269+0000", 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9982152, 2024-09-13T04:58:53.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010678615000000001, 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76980 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461269+0000", 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461269+0000", 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461268+0000", 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461270+0000", 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.432e-06 2024-09-13T04:58:53.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461273+0000", 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0170000000000001e-06 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461275+0000", 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5579999999999999e-06 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76789 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417508+0000", 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.041975851000000002, 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054545333000000001, 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76789 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.977 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417508+0000", 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417508+0000", 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417508+0000", 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417509+0000", 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3850000000000001e-06 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417512+0000", 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.038e-06 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417515+0000", 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.672e-06 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76821 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425236+0000", 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034247608999999998, 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.046923686999999999, 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.980 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.984 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76821 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425236+0000", 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425236+0000", 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425236+0000", 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425238+0000", 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425245+0000", 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3439999999999997e-06 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425251+0000", 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0260000000000004e-06 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76853 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429488+0000", 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.029996358000000001, 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042774870999999999, 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.987 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76853 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429488+0000", 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429488+0000", 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429487+0000", 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429489+0000", 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.815e-06 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429494+0000", 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8409999999999999e-06 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429497+0000", 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1949999999999999e-06 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76885 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437365+0000", 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022118593999999998, 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.034981486999999999, 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76885 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437365+0000", 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437365+0000", 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437365+0000", 2024-09-13T04:58:53.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437366+0000", 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3060000000000001e-06 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437369+0000", 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9629999999999998e-06 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437372+0000", 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6979999999999998e-06 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76917 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445177+0000", 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014306704999999999, 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027235473, 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.994 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76917 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445177+0000", 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445177+0000", 2024-09-13T04:58:53.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445177+0000", 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445178+0000", 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4169999999999999e-06 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:53.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445181+0000", 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0060000000000001e-06 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445184+0000", 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6089999999999999e-06 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.997 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76949 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453131+0000", 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0063525070000000003, 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019336116, 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:53.998 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76949 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453131+0000", 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453131+0000", 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:53.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453131+0000", 2024-09-13T04:58:54.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453133+0000", 2024-09-13T04:58:54.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.389e-06 2024-09-13T04:58:54.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453136+0000", 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.881e-06 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453138+0000", 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6110000000000001e-06 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.001 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76981 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461278+0000", 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9982061, 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011264336999999999, 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76981 2024-09-13T04:58:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461278+0000", 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461278+0000", 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461277+0000", 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461279+0000", 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3340000000000001e-06 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461282+0000", 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9979999999999999e-06 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461284+0000", 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5229999999999998e-06 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76790 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417517+0000", 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.041966732, 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055127145000000002, 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.006 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76790 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417517+0000", 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417517+0000", 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417517+0000", 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417518+0000", 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3400000000000001e-06 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417521+0000", 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9409999999999999e-06 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417524+0000", 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.756e-06 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76822 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425254+0000", 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034229532, 2024-09-13T04:58:54.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.047694356, 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76822 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425254+0000", 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425254+0000", 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425254+0000", 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425256+0000", 2024-09-13T04:58:54.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.283e-06 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425269+0000", 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2289e-05 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425275+0000", 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6030000000000001e-06 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76854 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429500+0000", 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.029984226999999999, 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043501501999999997, 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76854 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429500+0000", 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429500+0000", 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429500+0000", 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429501+0000", 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7069999999999999e-06 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429505+0000", 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0029999999999996e-06 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429508+0000", 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.852e-06 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.016 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76886 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437374+0000", 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022109567, 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035692719999999997, 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76886 2024-09-13T04:58:54.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437374+0000", 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437374+0000", 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437374+0000", 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437375+0000", 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.297e-06 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437378+0000", 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.88e-06 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437381+0000", 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7420000000000002e-06 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76918 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445186+0000", 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014297528, 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027982597000000001, 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.020 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76918 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445186+0000", 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445186+0000", 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445186+0000", 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445188+0000", 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3319999999999999e-06 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445191+0000", 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.959e-06 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445193+0000", 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6259999999999999e-06 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.023 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76950 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453140+0000", 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0063437270000000004, 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020144184999999998, 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76950 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453140+0000", 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453140+0000", 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453140+0000", 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453152+0000", 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1552999999999999e-05 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453156+0000", 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8459999999999999e-06 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453160+0000", 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1879999999999998e-06 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.026 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76982 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461286+0000", 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981976, 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012100276, 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.027 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76982 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461286+0000", 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461286+0000", 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461286+0000", 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461288+0000", 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.297e-06 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461291+0000", 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9749999999999999e-06 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461293+0000", 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.52e-06 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76791 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417527+0000", 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.041956912999999998, 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055955842999999998, 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76791 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417527+0000", 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417527+0000", 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417527+0000", 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417528+0000", 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5570000000000001e-06 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417531+0000", 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9280000000000002e-06 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417534+0000", 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5560000000000002e-06 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76823 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425280+0000", 2024-09-13T04:58:54.033 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034203883999999997, 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.04826855, 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76823 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.034 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425280+0000", 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425280+0000", 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425280+0000", 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425282+0000", 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425290+0000", 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.452e-06 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425295+0000", 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2179999999999998e-06 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76855 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.037 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429510+0000", 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.029973588999999998, 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.044149009000000003, 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76855 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.038 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429510+0000", 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429510+0000", 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429510+0000", 2024-09-13T04:58:54.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429511+0000", 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3239999999999999e-06 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429515+0000", 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9530000000000001e-06 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429519+0000", 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1360000000000001e-06 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.042 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76887 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.042 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437384+0000", 2024-09-13T04:58:54.042 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022100443000000001, 2024-09-13T04:58:54.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.036377484000000002, 2024-09-13T04:58:54.042 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.042 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76887 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437384+0000", 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437384+0000", 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437383+0000", 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437385+0000", 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3939999999999999e-06 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437388+0000", 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9569999999999998e-06 2024-09-13T04:58:54.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437390+0000", 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.729e-06 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76919 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445196+0000", 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014288329000000001, 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.028668542000000002, 2024-09-13T04:58:54.046 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76919 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445196+0000", 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445196+0000", 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445195+0000", 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445197+0000", 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3650000000000001e-06 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445200+0000", 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.92e-06 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445202+0000", 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6029999999999999e-06 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.049 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76951 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453162+0000", 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.006321747, 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.020808612000000001, 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.050 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76951 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453162+0000", 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453162+0000", 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453162+0000", 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453163+0000", 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.48e-06 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453166+0000", 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0290000000000001e-06 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453169+0000", 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6649999999999999e-06 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76983 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461295+0000", 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981885, 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01277936, 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76983 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461295+0000", 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461295+0000", 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461295+0000", 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461296+0000", 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.31e-06 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461299+0000", 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9720000000000001e-06 2024-09-13T04:58:54.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461302+0000", 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8420000000000001e-06 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76792 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417536+0000", 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.041947845999999997, 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056648242000000001, 2024-09-13T04:58:54.057 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76792 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417536+0000", 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417536+0000", 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417536+0000", 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417547+0000", 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1032e-05 2024-09-13T04:58:54.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417551+0000", 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3680000000000004e-06 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417554+0000", 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0709999999999999e-06 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76824 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425301+0000", 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034183382999999998, 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.048984933000000001, 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76824 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425301+0000", 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425301+0000", 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425300+0000", 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425302+0000", 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1670000000000002e-06 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425312+0000", 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2869999999999993e-06 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425319+0000", 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9210000000000003e-06 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.063 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76856 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429522+0000", 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.029962299000000001, 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.044871510000000003, 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76856 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.064 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429522+0000", 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429522+0000", 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429521+0000", 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429523+0000", 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3069999999999999e-06 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429527+0000", 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0439999999999997e-06 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429530+0000", 2024-09-13T04:58:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9969999999999999e-06 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76888 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437393+0000", 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022091372000000001, 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037103432999999998, 2024-09-13T04:58:54.067 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76888 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437393+0000", 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437393+0000", 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437392+0000", 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437394+0000", 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.336e-06 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437397+0000", 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9220000000000001e-06 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437399+0000", 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.774e-06 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76920 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445205+0000", 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014279408, 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029397512000000001, 2024-09-13T04:58:54.071 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76920 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445205+0000", 2024-09-13T04:58:54.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445205+0000", 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445204+0000", 2024-09-13T04:58:54.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445206+0000", 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3120000000000001e-06 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445209+0000", 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8669999999999998e-06 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445211+0000", 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.604e-06 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.075 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76952 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453171+0000", 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0063127280000000001, 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021533170000000001, 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76952 2024-09-13T04:58:54.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453171+0000", 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453171+0000", 2024-09-13T04:58:54.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453171+0000", 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453172+0000", 2024-09-13T04:58:54.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.314e-06 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453175+0000", 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9579999999999999e-06 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453178+0000", 2024-09-13T04:58:54.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6199999999999999e-06 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76984 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461304+0000", 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981794, 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013508275, 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.080 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76984 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461304+0000", 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461304+0000", 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461304+0000", 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461305+0000", 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2359999999999999e-06 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461308+0000", 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0130000000000002e-06 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461311+0000", 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5950000000000001e-06 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.083 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76793 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417557+0000", 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.041927043999999997, 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057359186999999999, 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76793 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417557+0000", 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417557+0000", 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417557+0000", 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417558+0000", 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3969999999999999e-06 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417561+0000", 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.999e-06 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417564+0000", 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2289999999999999e-06 2024-09-13T04:58:54.086 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76825 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425323+0000", 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034160691, 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.049701863999999998, 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.087 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76825 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425323+0000", 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425323+0000", 2024-09-13T04:58:54.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425323+0000", 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425325+0000", 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4870000000000001e-06 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425333+0000", 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1760000000000004e-06 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425337+0000", 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.887e-06 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76857 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.090 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.429533+0000", 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.029951127000000001, 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.04559738, 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76857 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429533+0000", 2024-09-13T04:58:54.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429533+0000", 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429533+0000", 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429534+0000", 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.344e-06 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429537+0000", 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0620000000000001e-06 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429540+0000", 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6469999999999998e-06 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.093 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76889 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437402+0000", 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022082114999999999, 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037830857000000002, 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76889 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.094 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437402+0000", 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437402+0000", 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437401+0000", 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437406+0000", 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5299999999999998e-06 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437409+0000", 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0639999999999998e-06 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437412+0000", 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3419999999999999e-06 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76921 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445214+0000", 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014270448, 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030090907, 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76921 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445214+0000", 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445214+0000", 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445213+0000", 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445214+0000", 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1909999999999999e-06 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445218+0000", 2024-09-13T04:58:54.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6370000000000001e-06 2024-09-13T04:58:54.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445221+0000", 2024-09-13T04:58:54.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7080000000000002e-06 2024-09-13T04:58:54.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76953 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453180+0000", 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0063037500000000003, 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.022175278999999999, 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.101 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76953 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453180+0000", 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453180+0000", 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453180+0000", 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453181+0000", 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.407e-06 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453185+0000", 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7519999999999998e-06 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453188+0000", 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.61e-06 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76985 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461313+0000", 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981709, 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014103221000000001, 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76985 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461313+0000", 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461313+0000", 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461313+0000", 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461314+0000", 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.173e-06 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461317+0000", 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9500000000000001e-06 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461320+0000", 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5390000000000002e-06 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76794 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.417567+0000", 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.041917287999999997, 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057943630000000003, 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.108 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76794 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417567+0000", 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417567+0000", 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417566+0000", 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417568+0000", 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4670000000000001e-06 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417573+0000", 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8960000000000002e-06 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417576+0000", 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.728e-06 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76826 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425342+0000", 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034142389000000002, 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050275383, 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.112 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76826 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425342+0000", 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425342+0000", 2024-09-13T04:58:54.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425341+0000", 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425344+0000", 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7810000000000001e-06 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425352+0000", 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2789999999999998e-06 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425356+0000", 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4570000000000001e-06 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76858 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433067+0000", 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026416861, 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.042660690000000001, 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.116 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.117 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76858 2024-09-13T04:58:54.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.117 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433067+0000", 2024-09-13T04:58:54.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433067+0000", 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.429542+0000", 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9964747 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433071+0000", 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035288110000000002 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433077+0000", 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9900000000000002e-06 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433080+0000", 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7409999999999998e-06 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76890 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.437416+0000", 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.022068355000000001, 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038419174, 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76890 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.120 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437416+0000", 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437416+0000", 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437415+0000", 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437417+0000", 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4479999999999999e-06 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437421+0000", 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0260000000000001e-06 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437424+0000", 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7369999999999998e-06 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76922 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445223+0000", 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014260851999999999, 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030714637999999999, 2024-09-13T04:58:54.123 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76922 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445223+0000", 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445223+0000", 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445223+0000", 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445224+0000", 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3960000000000001e-06 2024-09-13T04:58:54.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445227+0000", 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9629999999999998e-06 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445230+0000", 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1839999999999999e-06 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.126 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76954 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453190+0000", 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0062941250000000002, 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.022854935, 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76954 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.127 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453190+0000", 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453190+0000", 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453190+0000", 2024-09-13T04:58:54.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453191+0000", 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3859999999999999e-06 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453195+0000", 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5319999999999998e-06 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453197+0000", 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.723e-06 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76986 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461322+0000", 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981623, 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014825741999999999, 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.130 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76986 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461322+0000", 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461322+0000", 2024-09-13T04:58:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461322+0000", 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461327+0000", 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2360000000000004e-06 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461335+0000", 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7700000000000001e-06 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461338+0000", 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3720000000000001e-06 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76795 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421133+0000", 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038350634000000002, 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055123837000000002, 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76795 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421133+0000", 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421133+0000", 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.417583+0000", 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9964499 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421138+0000", 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0035551630000000001 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421150+0000", 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1324e-05 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421158+0000", 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.884e-06 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.136 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76827 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425359+0000", 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034125338999999998, 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.051002058000000003, 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76827 2024-09-13T04:58:54.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425359+0000", 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425359+0000", 2024-09-13T04:58:54.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425358+0000", 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425360+0000", 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.39e-06 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425363+0000", 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2490000000000001e-06 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425367+0000", 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1389999999999997e-06 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76859 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433110+0000", 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026374254999999999, 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043353461000000003, 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.141 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76859 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433110+0000", 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433110+0000", 2024-09-13T04:58:54.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433109+0000", 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433111+0000", 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.897e-06 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433121+0000", 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433124+0000", 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1520000000000001e-06 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.144 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76891 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441183+0000", 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018301367999999998, 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035391635999999997, 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76891 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441183+0000", 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441183+0000", 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.437435+0000", 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9962521 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441186+0000", 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0037511530000000001 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441193+0000", 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0360000000000001e-06 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441196+0000", 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6959999999999998e-06 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.147 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76923 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445233+0000", 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014250783, 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031443589000000001, 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.148 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76923 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445233+0000", 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445233+0000", 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445233+0000", 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445234+0000", 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3960000000000001e-06 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445238+0000", 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1709999999999998e-06 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445240+0000", 2024-09-13T04:58:54.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.65e-06 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76955 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453200+0000", 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.006284441, 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.023584603999999999, 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.152 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.153 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76955 2024-09-13T04:58:54.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.153 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453200+0000", 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453200+0000", 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453199+0000", 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453201+0000", 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.344e-06 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453204+0000", 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9689999999999999e-06 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453206+0000", 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6000000000000001e-06 2024-09-13T04:58:54.155 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76987 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461341+0000", 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981432, 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015546127999999999, 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.156 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76987 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461341+0000", 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461341+0000", 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461340+0000", 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461342+0000", 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.415e-06 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461345+0000", 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0589999999999998e-06 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461347+0000", 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6350000000000002e-06 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76796 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421167+0000", 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038316954, 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055829978000000002, 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.159 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76796 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421167+0000", 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421167+0000", 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421166+0000", 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421169+0000", 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9749999999999999e-06 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421176+0000", 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2029999999999997e-06 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421180+0000", 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8489999999999997e-06 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.162 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76828 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425370+0000", 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034113816999999998, 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.051739775000000002, 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76828 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425370+0000", 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425370+0000", 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425370+0000", 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425375+0000", 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3059999999999997e-06 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425378+0000", 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3170000000000002e-06 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425381+0000", 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.785e-06 2024-09-13T04:58:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76860 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433127+0000", 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026357187000000001, 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.044061859000000002, 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.166 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76860 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433127+0000", 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433127+0000", 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433127+0000", 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433128+0000", 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.601e-06 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433131+0000", 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2440000000000002e-06 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433134+0000", 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.756e-06 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76892 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441200+0000", 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018283822000000002, 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.036039100999999997, 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76892 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441200+0000", 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441200+0000", 2024-09-13T04:58:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441200+0000", 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441201+0000", 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6640000000000001e-06 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441205+0000", 2024-09-13T04:58:54.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0949999999999998e-06 2024-09-13T04:58:54.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441208+0000", 2024-09-13T04:58:54.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7130000000000001e-06 2024-09-13T04:58:54.172 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.172 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76924 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.445242+0000", 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.014241487000000001, 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.032061736, 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76924 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445242+0000", 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445242+0000", 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445242+0000", 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445244+0000", 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3459999999999999e-06 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445252+0000", 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2970000000000004e-06 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445255+0000", 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.943e-06 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.176 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76956 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453210+0000", 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0062741150000000002, 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024193978000000001, 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76956 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.177 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453210+0000", 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453210+0000", 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453210+0000", 2024-09-13T04:58:54.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453211+0000", 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.223e-06 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453219+0000", 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9100000000000005e-06 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453222+0000", 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0850000000000001e-06 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76988 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461350+0000", 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981341, 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016156764000000001, 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.180 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76988 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461350+0000", 2024-09-13T04:58:54.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461350+0000", 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461349+0000", 2024-09-13T04:58:54.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461351+0000", 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.252e-06 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461354+0000", 2024-09-13T04:58:54.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9220000000000001e-06 2024-09-13T04:58:54.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461356+0000", 2024-09-13T04:58:54.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4770000000000002e-06 2024-09-13T04:58:54.184 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.184 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.184 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.185 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76797 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.188 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421185+0000", 2024-09-13T04:58:54.188 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038298956000000002, 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.056430726, 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76797 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421185+0000", 2024-09-13T04:58:54.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421185+0000", 2024-09-13T04:58:54.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421185+0000", 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421187+0000", 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2649999999999999e-06 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421193+0000", 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7919999999999998e-06 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421197+0000", 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7690000000000004e-06 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76829 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.192 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425384+0000", 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034099834000000002, 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052332732, 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76829 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425384+0000", 2024-09-13T04:58:54.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425384+0000", 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425384+0000", 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425385+0000", 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.215e-06 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425388+0000", 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0489999999999999e-06 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425391+0000", 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6410000000000002e-06 2024-09-13T04:58:54.195 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76861 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433136+0000", 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026347588000000002, 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.044687108000000003, 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.196 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76861 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433136+0000", 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433136+0000", 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433136+0000", 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433138+0000", 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3880000000000001e-06 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433141+0000", 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.038e-06 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433143+0000", 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.553e-06 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76893 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441218+0000", 2024-09-13T04:58:54.199 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018265482, 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.036707206999999999, 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76893 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441218+0000", 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441218+0000", 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441218+0000", 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441220+0000", 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.579e-06 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441228+0000", 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3149999999999992e-06 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441231+0000", 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8739999999999999e-06 2024-09-13T04:58:54.202 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76925 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449172+0000", 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010312389, 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.028845125999999999, 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.203 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76925 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449172+0000", 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449172+0000", 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.445257+0000", 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9960856 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449178+0000", 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0039203170000000004 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449186+0000", 2024-09-13T04:58:54.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5429999999999994e-06 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449190+0000", 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9659999999999998e-06 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76957 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453224+0000", 2024-09-13T04:58:54.206 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0062597060000000003, 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.024845596000000001, 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76957 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453224+0000", 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453224+0000", 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453224+0000", 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453225+0000", 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.401e-06 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453229+0000", 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6739999999999999e-06 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453232+0000", 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0699999999999998e-06 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76989 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461359+0000", 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981251, 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016773825999999999, 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76989 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.210 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461359+0000", 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461359+0000", 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461358+0000", 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461360+0000", 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.325e-06 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461363+0000", 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.039e-06 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461366+0000", 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1729999999999999e-06 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76798 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421202+0000", 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038282104999999997, 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057036479000000001, 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.214 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76798 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421202+0000", 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421202+0000", 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421201+0000", 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421204+0000", 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.323e-06 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421213+0000", 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4490000000000005e-06 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421219+0000", 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3399999999999997e-06 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76830 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425393+0000", 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034090991000000001, 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.052953053, 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.218 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76830 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425393+0000", 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425393+0000", 2024-09-13T04:58:54.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425393+0000", 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425394+0000", 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.311e-06 2024-09-13T04:58:54.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425397+0000", 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9840000000000001e-06 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425400+0000", 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0010000000000002e-06 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.222 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.222 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76862 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.222 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433146+0000", 2024-09-13T04:58:54.222 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026338002999999999, 2024-09-13T04:58:54.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.045301973000000002, 2024-09-13T04:58:54.222 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.223 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.223 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76862 2024-09-13T04:58:54.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.223 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433146+0000", 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433146+0000", 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433146+0000", 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433147+0000", 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4169999999999999e-06 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433151+0000", 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1880000000000004e-06 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433155+0000", 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3079999999999999e-06 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76894 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441234+0000", 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018250394, 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037321879000000002, 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.226 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76894 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441234+0000", 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441234+0000", 2024-09-13T04:58:54.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441233+0000", 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441235+0000", 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4050000000000001e-06 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441239+0000", 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1400000000000002e-06 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441242+0000", 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6960000000000001e-06 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76926 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449218+0000", 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010266437, 2024-09-13T04:58:54.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029440253999999999, 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76926 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449218+0000", 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449218+0000", 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449217+0000", 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449219+0000", 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.784e-06 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449223+0000", 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.715e-06 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449226+0000", 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8499999999999998e-06 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.232 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76958 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453234+0000", 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0062494930000000001, 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025525566, 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76958 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453234+0000", 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453234+0000", 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453234+0000", 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453236+0000", 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.347e-06 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453239+0000", 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9529999999999999e-06 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453242+0000", 2024-09-13T04:58:54.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4450000000000001e-06 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76990 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461376+0000", 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9981079, 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.017490287, 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.236 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76990 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461376+0000", 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461376+0000", 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461376+0000", 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461377+0000", 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.562e-06 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461382+0000", 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.476e-06 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461385+0000", 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7530000000000002e-06 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76799 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421223+0000", 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038260770999999999, 2024-09-13T04:58:54.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.057749515000000001, 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76799 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421223+0000", 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421223+0000", 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421223+0000", 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421225+0000", 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5220000000000002e-06 2024-09-13T04:58:54.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421234+0000", 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7250000000000003e-06 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421241+0000", 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7800000000000003e-06 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76831 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425402+0000", 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034081686999999999, 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.053678653999999999, 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76831 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425402+0000", 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425402+0000", 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425402+0000", 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425403+0000", 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.283e-06 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425409+0000", 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2329999999999997e-06 2024-09-13T04:58:54.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425411+0000", 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7420000000000002e-06 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76863 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433157+0000", 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026326840000000001, 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.046024876999999999, 2024-09-13T04:58:54.246 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76863 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433157+0000", 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433157+0000", 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433157+0000", 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433158+0000", 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.483e-06 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433161+0000", 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0759999999999999e-06 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433164+0000", 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6570000000000001e-06 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76895 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441244+0000", 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018240036000000001, 2024-09-13T04:58:54.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038044730999999998, 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76895 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441244+0000", 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441244+0000", 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441244+0000", 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441245+0000", 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.421e-06 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441248+0000", 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0110000000000001e-06 2024-09-13T04:58:54.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441251+0000", 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.695e-06 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76927 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449228+0000", 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010256095999999999, 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030163720000000002, 2024-09-13T04:58:54.254 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76927 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449228+0000", 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449228+0000", 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449228+0000", 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449239+0000", 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1362000000000001e-05 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449244+0000", 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7129999999999996e-06 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449246+0000", 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6630000000000002e-06 2024-09-13T04:58:54.257 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76959 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453244+0000", 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0062395480000000001, 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026253584, 2024-09-13T04:58:54.258 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.259 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.259 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.259 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.259 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.259 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76959 2024-09-13T04:58:54.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.259 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453244+0000", 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453244+0000", 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453244+0000", 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453245+0000", 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.218e-06 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453249+0000", 2024-09-13T04:58:54.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.76e-06 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453252+0000", 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6060000000000001e-06 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76991 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461388+0000", 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9980965, 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018213414000000001, 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.262 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76991 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461388+0000", 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461388+0000", 2024-09-13T04:58:54.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461387+0000", 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461389+0000", 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4920000000000001e-06 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461392+0000", 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9730000000000002e-06 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461395+0000", 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2990000000000001e-06 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76800 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.265 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421245+0000", 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038238762000000003, 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.058469240999999998, 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76800 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421245+0000", 2024-09-13T04:58:54.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421245+0000", 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421245+0000", 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421247+0000", 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.277e-06 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421254+0000", 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7440000000000001e-06 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421259+0000", 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8250000000000004e-06 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.268 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76832 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425414+0000", 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034070216, 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054423244000000003, 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.269 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76832 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425414+0000", 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425414+0000", 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425414+0000", 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425415+0000", 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.26e-06 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425418+0000", 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.129e-06 2024-09-13T04:58:54.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425421+0000", 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.773e-06 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76864 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433166+0000", 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026317736000000001, 2024-09-13T04:58:54.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.046731223000000002, 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76864 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433166+0000", 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433166+0000", 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433166+0000", 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433167+0000", 2024-09-13T04:58:54.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.341e-06 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433170+0000", 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9179999999999998e-06 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433173+0000", 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6740000000000001e-06 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76896 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441254+0000", 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018229908, 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.038693942000000002, 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76896 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441254+0000", 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441254+0000", 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441254+0000", 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441255+0000", 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2810000000000001e-06 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441258+0000", 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9620000000000002e-06 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441261+0000", 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.632e-06 2024-09-13T04:58:54.278 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76928 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449249+0000", 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010234966, 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.030749004999999999, 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76928 2024-09-13T04:58:54.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449249+0000", 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449249+0000", 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449249+0000", 2024-09-13T04:58:54.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449250+0000", 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.553e-06 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449253+0000", 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9909999999999998e-06 2024-09-13T04:58:54.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449256+0000", 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7439999999999999e-06 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76960 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453254+0000", 2024-09-13T04:58:54.282 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0062298140000000002, 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026796783000000001, 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76960 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453254+0000", 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453254+0000", 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453254+0000", 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453255+0000", 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.35e-06 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453258+0000", 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.898e-06 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453261+0000", 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7190000000000001e-06 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76992 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461397+0000", 2024-09-13T04:58:54.286 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9980869, 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.018712696000000001, 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76992 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461397+0000", 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461397+0000", 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461397+0000", 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461398+0000", 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.33e-06 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461402+0000", 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6030000000000001e-06 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461405+0000", 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.65e-06 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76801 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421263+0000", 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038221032000000002, 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.058954015999999998, 2024-09-13T04:58:54.290 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76801 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421263+0000", 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421263+0000", 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421263+0000", 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421265+0000", 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.221e-06 2024-09-13T04:58:54.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421271+0000", 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8039999999999998e-06 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421276+0000", 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9779999999999996e-06 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76833 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425423+0000", 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034060967999999997, 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.054897546999999998, 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76833 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425423+0000", 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425423+0000", 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425423+0000", 2024-09-13T04:58:54.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425424+0000", 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.226e-06 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425427+0000", 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9560000000000002e-06 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425430+0000", 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7030000000000002e-06 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76865 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433175+0000", 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026308934999999999, 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.047251464, 2024-09-13T04:58:54.297 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76865 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433175+0000", 2024-09-13T04:58:54.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433175+0000", 2024-09-13T04:58:54.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433175+0000", 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433180+0000", 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9679999999999997e-06 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433187+0000", 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1760000000000004e-06 2024-09-13T04:58:54.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433190+0000", 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1609999999999999e-06 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76897 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441263+0000", 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018220975, 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039263800000000001, 2024-09-13T04:58:54.301 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76897 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441263+0000", 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441263+0000", 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441263+0000", 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441264+0000", 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2419999999999999e-06 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441267+0000", 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9160000000000001e-06 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441270+0000", 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3689999999999998e-06 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76929 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449258+0000", 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010225730000000001, 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.031375472000000001, 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.305 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76929 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449258+0000", 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449258+0000", 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449258+0000", 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449259+0000", 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3170000000000001e-06 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449262+0000", 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9909999999999998e-06 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449265+0000", 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5969999999999999e-06 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76961 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453291+0000", 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0061928010000000004, 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027444447, 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76961 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453291+0000", 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453291+0000", 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453291+0000", 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453293+0000", 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5100000000000001e-06 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453298+0000", 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3120000000000004e-06 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453301+0000", 2024-09-13T04:58:54.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0019999999999998e-06 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76993 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.461407+0000", 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9980774, 2024-09-13T04:58:54.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019436592999999999, 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76993 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461407+0000", 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461407+0000", 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461406+0000", 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461408+0000", 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.268e-06 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461411+0000", 2024-09-13T04:58:54.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9739999999999998e-06 2024-09-13T04:58:54.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461413+0000", 2024-09-13T04:58:54.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.542e-06 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76802 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.421281+0000", 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.038203147, 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059667114, 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.316 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76802 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421281+0000", 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421281+0000", 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421280+0000", 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421283+0000", 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421288+0000", 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.665e-06 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.421293+0000", 2024-09-13T04:58:54.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2869999999999998e-06 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76834 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.425432+0000", 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.034051969000000001, 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.055627551999999997, 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.320 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76834 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425432+0000", 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425432+0000", 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425432+0000", 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425433+0000", 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.226e-06 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425436+0000", 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0249999999999998e-06 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.425439+0000", 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7070000000000001e-06 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76866 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.433193+0000", 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.026290581, 2024-09-13T04:58:54.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.047972985000000003, 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76866 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433193+0000", 2024-09-13T04:58:54.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433193+0000", 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433193+0000", 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433195+0000", 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4109999999999999e-06 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433202+0000", 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.323e-06 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.433205+0000", 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7070000000000001e-06 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.327 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76898 3.19 3.8154ebb9 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.441272+0000", 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.018211484999999999, 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.039946453999999999, 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.329 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76898 2024-09-13T04:58:54.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.329 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441272+0000", 2024-09-13T04:58:54.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441272+0000", 2024-09-13T04:58:54.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441272+0000", 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441274+0000", 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2550000000000001e-06 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441277+0000", 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.614e-06 2024-09-13T04:58:54.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.441280+0000", 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0060000000000001e-06 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76930 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.449267+0000", 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.010216909999999999, 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.032005928000000003, 2024-09-13T04:58:54.332 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76930 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449267+0000", 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449267+0000", 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449267+0000", 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449268+0000", 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.294e-06 2024-09-13T04:58:54.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449271+0000", 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9380000000000001e-06 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.449274+0000", 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5900000000000002e-06 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76962 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.453303+0000", 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0061809869999999998, 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.028021414000000001, 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76962 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453303+0000", 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453303+0000", 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453303+0000", 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967296 2024-09-13T04:58:54.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453312+0000", 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6309999999999998e-06 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453317+0000", 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6609999999999999e-06 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.453320+0000", 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8329999999999998e-06 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.338 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:76994 3.19 3.8154ebb9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.465088+0000", 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9943957, 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.016289482000000001, 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 76994 2024-09-13T04:58:54.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.465088+0000", 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.465088+0000", 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.461415+0000", 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9963274 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.465091+0000", 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.003675832 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.465098+0000", 2024-09-13T04:58:54.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7170000000000002e-06 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.465102+0000", 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.89e-06 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 230 2024-09-13T04:58:54.342 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:54.343 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:58:54.345 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T04:58:54.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:54.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T04:58:54.346 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:54.339+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:54.442 INFO:teuthology.orchestra.run.smithi001.stdout:2785 flatten 2024-09-13T04:58:54.444 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:54.443+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:54.488 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:54.488 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:54.489 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:54.489 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:54.489 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:54.513 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:54.557 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:54.555+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:54.597 INFO:teuthology.orchestra.run.smithi001.stdout:2786 write 0x8075082 thru 0x80785e7 (0x3566 bytes) 2024-09-13T04:58:54.601 INFO:teuthology.orchestra.run.smithi001.stdout:2788 read 0x267b1bd thru 0x2684505 (0x9349 bytes) 2024-09-13T04:58:54.611 INFO:teuthology.orchestra.run.smithi001.stdout:2789 write 0xbce67ea thru 0xbcea738 (0x3f4f bytes) 2024-09-13T04:58:54.615 INFO:teuthology.orchestra.run.smithi001.stdout:2790 write 0x7758f06 thru 0x7768264 (0xf35f bytes) 2024-09-13T04:58:54.621 INFO:teuthology.orchestra.run.smithi001.stdout:2791 trunc from 0xbcea739 to 0xba06449 2024-09-13T04:58:54.640 INFO:teuthology.orchestra.run.smithi001.stdout:2792 trunc from 0xba06449 to 0x5f3f464 2024-09-13T04:58:54.646 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:54.643+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:54.668 INFO:teuthology.orchestra.run.smithi001.stdout:2793 trunc from 0x5f3f464 to 0xaf80bb9 2024-09-13T04:58:54.676 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:54.677 INFO:teuthology.orchestra.run.smithi001.stdout:2795 write 0x2a3cbc9 thru 0x2a44921 (0x7d59 bytes) 2024-09-13T04:58:54.688 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.82859156300000003, 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T04:58:54.689 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T04:58:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T04:58:54.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.82856465199999996, 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.695 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T04:58:54.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T04:58:54.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:58:54.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T04:58:54.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T04:58:54.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.700 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.82481901599999996, 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.701 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T04:58:54.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T04:58:54.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T04:58:54.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.82127569099999997, 2024-09-13T04:58:54.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T04:58:54.707 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.707 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.707 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T04:58:54.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T04:58:54.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T04:58:54.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.82123606299999996, 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T04:58:54.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:58:54.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T04:58:54.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T04:58:54.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.82121240299999998, 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.718 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:58:54.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:58:54.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T04:58:54.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:58:54.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T04:58:54.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.723 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.82112485400000002, 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T04:58:54.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T04:58:54.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T04:58:54.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T04:58:54.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T04:58:54.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.729 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.729 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.729 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.729 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:58:54.729 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.82107219799999998, 2024-09-13T04:58:54.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:58:54.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:58:54.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T04:58:54.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:54.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T04:58:54.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T04:58:54.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T04:58:54.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T04:58:54.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T04:58:54.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T04:58:54.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T04:58:54.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T04:58:54.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T04:58:54.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T04:58:54.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T04:58:54.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T04:58:54.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.737 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.738 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.738 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:58:54.738 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.81714630700000002, 2024-09-13T04:58:54.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T04:58:54.738 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.739 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.739 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.739 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.739 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.739 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T04:58:54.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:58:54.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:58:54.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T04:58:54.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T04:58:54.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T04:58:54.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T04:58:54.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T04:58:54.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T04:58:54.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T04:58:54.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T04:58:54.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T04:58:54.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T04:58:54.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T04:58:54.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T04:58:54.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.748 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.748 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.81712068199999999, 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T04:58:54.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.750 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:58:54.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T04:58:54.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T04:58:54.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.81708728799999997, 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T04:58:54.755 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T04:58:54.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T04:58:54.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.812297359, 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.761 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.761 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T04:58:54.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.761 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:58:54.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:58:54.765 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.80890273000000001, 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T04:58:54.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:54.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T04:58:54.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T04:58:54.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T04:58:54.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T04:58:54.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T04:58:54.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T04:58:54.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T04:58:54.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T04:58:54.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T04:58:54.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T04:58:54.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T04:58:54.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T04:58:54.772 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.772 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.773 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.773 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:58:54.773 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.80883005399999996, 2024-09-13T04:58:54.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T04:58:54.774 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.774 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.774 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T04:58:54.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.775 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:58:54.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:58:54.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T04:58:54.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.80508769300000005, 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.780 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T04:58:54.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T04:58:54.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.80491470200000004, 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T04:58:54.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.80107210299999998, 2024-09-13T04:58:54.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T04:58:54.790 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.790 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.790 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T04:58:54.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.790 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:58:54.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T04:58:54.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:58:54.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T04:58:54.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.795 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.76122412500000003, 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T04:58:54.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T04:58:54.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:58:54.800 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.75689490299999995, 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:54.805 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:58:54.808 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.74916590299999997, 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T04:58:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:54.813 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:54.816 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:54.816 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:54.811+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:2796 write 0xe9a64d6 thru 0xe9b45bd (0xe0e8 bytes) 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:2799 write 0x9d42921 thru 0x9d44743 (0x1e23 bytes) 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:2800 read 0xd0347b thru 0xd03f64 (0xaea bytes) 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:2801 writesame 0x6a0b27f thru 0x6a16e91 data_size 0xb(0xbc13 bytes) 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:2802 write 0x743ffb6 thru 0x7448be9 (0x8c34 bytes) 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:2806 trunc from 0xe9b45be to 0x3685c3c 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:2807 read 0x1874b45 thru 0x18806ad (0xbb69 bytes) 2024-09-13T04:58:54.817 INFO:teuthology.orchestra.run.smithi001.stdout:2809 write 0x58c3d77 thru 0x58cd72d (0x99b7 bytes) 2024-09-13T04:58:54.831 INFO:teuthology.orchestra.run.smithi001.stdout:2810 write 0xafe2818 thru 0xafe8d84 (0x656d bytes) 2024-09-13T04:58:54.839 INFO:teuthology.orchestra.run.smithi001.stdout:2811 writesame 0x77457a2 thru 0x774e7cf data_size 0x5(0x902e bytes) 2024-09-13T04:58:54.870 INFO:teuthology.orchestra.run.smithi001.stdout:2812 trunc from 0xafe8d85 to 0x62ebaa2 2024-09-13T04:58:54.902 INFO:teuthology.orchestra.run.smithi001.stdout:2813 write 0x1cb5996 thru 0x1cbe684 (0x8cef bytes) 2024-09-13T04:58:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:58:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T04:58:54.910 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:54.910 INFO:teuthology.orchestra.run.smithi001.stdout:2817 read 0x335a57e thru 0x3366735 (0xc1b8 bytes) 2024-09-13T04:58:54.915 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:54.911+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:54.924 INFO:teuthology.orchestra.run.smithi001.stdout:2819 write 0x75138b3 thru 0x7516ac0 (0x320e bytes) 2024-09-13T04:58:54.933 INFO:teuthology.orchestra.run.smithi001.stdout:2821 read 0x631e1b8 thru 0x6324a05 (0x684e bytes) 2024-09-13T04:58:54.933 INFO:teuthology.orchestra.run.smithi001.stdout:2822 read 0x16001b0 thru 0x160c3f9 (0xc24a bytes) 2024-09-13T04:58:54.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T04:58:54.950 INFO:teuthology.orchestra.run.smithi001.stdout:2823 write 0xc793d4b thru 0xc7a0938 (0xcbee bytes) 2024-09-13T04:58:54.955 INFO:teuthology.orchestra.run.smithi001.stdout:2824 read 0xbf516ed thru 0xbf53cbf (0x25d3 bytes) 2024-09-13T04:58:54.955 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:54.955 INFO:teuthology.orchestra.run.smithi001.stdout:2827 punch from 0xb452284 to 0xb454c60, (0x29dc bytes) 2024-09-13T04:58:54.955 INFO:teuthology.orchestra.run.smithi001.stdout:2828 write 0x8563dd8 thru 0x857334a (0xf573 bytes) 2024-09-13T04:58:54.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:54.963 INFO:teuthology.orchestra.run.smithi001.stdout:2829 writesame 0x10cab4a thru 0x10d5ff4 data_size 0x1b(0xb4ab bytes) 2024-09-13T04:58:54.967 INFO:teuthology.orchestra.run.smithi001.stdout:2830 write 0x375a0f5 thru 0x3760845 (0x6751 bytes) 2024-09-13T04:58:54.970 INFO:teuthology.orchestra.run.smithi001.stdout:2831 punch from 0xc35e633 to 0xc35e825, (0x1f2 bytes) 2024-09-13T04:58:54.970 INFO:teuthology.orchestra.run.smithi001.stdout:2833 read 0x29d1daa thru 0x29d8324 (0x657b bytes) 2024-09-13T04:58:54.971 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:54.971 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:54.971 INFO:teuthology.orchestra.run.smithi001.stdout:2836 read 0xb9cb114 thru 0xb9dae42 (0xfd2f bytes) 2024-09-13T04:58:54.971 INFO:teuthology.orchestra.run.smithi001.stdout:2837 punch from 0x846698e to 0x8474431, (0xdaa3 bytes) 2024-09-13T04:58:54.987 INFO:teuthology.orchestra.run.smithi001.stdout:2838 trunc from 0xc7a0939 to 0x265099b 2024-09-13T04:58:54.999 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:54.999 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:54.999 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:55.000 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:55.016 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:55.017 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:55.015+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:55.025 INFO:teuthology.orchestra.run.smithi001.stdout:2840 write 0x63f103b thru 0x63f7391 (0x6357 bytes) 2024-09-13T04:58:55.030 INFO:teuthology.orchestra.run.smithi001.stdout:2841 write 0x3f01b27 thru 0x3f0f965 (0xde3f bytes) 2024-09-13T04:58:55.050 INFO:teuthology.orchestra.run.smithi001.stdout:2842 write 0x6e23126 thru 0x6e27be0 (0x4abb bytes) 2024-09-13T04:58:55.054 INFO:teuthology.orchestra.run.smithi001.stdout:2843 write 0x49317c5 thru 0x493d7d1 (0xc00d bytes) 2024-09-13T04:58:55.064 INFO:teuthology.orchestra.run.smithi001.stdout:2844 writesame 0x6776299 thru 0x677de62 data_size 0x5(0x7bca bytes) 2024-09-13T04:58:55.074 INFO:teuthology.orchestra.run.smithi001.stdout:2845 write 0x2a028ff thru 0x2a08693 (0x5d95 bytes) 2024-09-13T04:58:55.077 INFO:teuthology.orchestra.run.smithi001.stdout:2846 punch from 0x1dd9190 to 0x1dda2e7, (0x1157 bytes) 2024-09-13T04:58:55.094 INFO:teuthology.orchestra.run.smithi001.stdout:2847 write 0xc28b934 thru 0xc297022 (0xb6ef bytes) 2024-09-13T04:58:55.115 INFO:teuthology.orchestra.run.smithi001.stdout:2848 trunc from 0xc297023 to 0xeba92d 2024-09-13T04:58:55.116 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:55.111+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:55.128 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:55.133 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.147 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:55.147 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:55.154 INFO:teuthology.orchestra.run.smithi001.stdout:2851 writesame 0xb539e95 thru 0xb548c8e data_size 0x53(0xedfa bytes) 2024-09-13T04:58:55.164 INFO:teuthology.orchestra.run.smithi001.stdout:2852 writesame 0xd2cf930 thru 0xd2de5f6 data_size 0x2d(0xecc7 bytes) 2024-09-13T04:58:55.167 INFO:teuthology.orchestra.run.smithi001.stdout:2854 trunc from 0xd2de5f7 to 0xb2b8cfc 2024-09-13T04:58:55.168 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:55.168 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:55.175 INFO:teuthology.orchestra.run.smithi001.stdout:2855 read 0x49962be thru 0x499e7d8 (0x851b bytes) 2024-09-13T04:58:55.175 INFO:teuthology.orchestra.run.smithi001.stdout:2856 punch from 0x8da913a to 0x8daed0d, (0x5bd3 bytes) 2024-09-13T04:58:55.175 INFO:teuthology.orchestra.run.smithi001.stdout:2858 punch from 0x65c26ad to 0x65c2df2, (0x745 bytes) 2024-09-13T04:58:55.175 INFO:teuthology.orchestra.run.smithi001.stdout:2859 write 0x23b5d1c thru 0x23befc9 (0x92ae bytes) 2024-09-13T04:58:55.181 INFO:teuthology.orchestra.run.smithi001.stdout:2860 write 0x51096d2 thru 0x510b83a (0x2169 bytes) 2024-09-13T04:58:55.185 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:55.189 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:55.189 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:55.189 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:55.203 INFO:teuthology.orchestra.run.smithi001.stdout:2861 trunc from 0xb2b8cfc to 0x10bed6c 2024-09-13T04:58:55.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:55.215 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:55.215 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:55.217 INFO:teuthology.orchestra.run.smithi001.stdout:2862 read 0xeff4e0 thru 0xf0a8c3 (0xb3e4 bytes) 2024-09-13T04:58:55.222 INFO:teuthology.orchestra.run.smithi001.stdout:2864 trunc from 0x10bed6c to 0x50a04fb 2024-09-13T04:58:55.229 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.229 INFO:teuthology.orchestra.run.smithi001.stdout:2867 read 0x9e0573 thru 0x9f0507 (0xff95 bytes) 2024-09-13T04:58:55.234 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:55.241 INFO:teuthology.orchestra.run.smithi001.stdout:2868 write 0x976bfc1 thru 0x97788a5 (0xc8e5 bytes) 2024-09-13T04:58:55.245 INFO:teuthology.orchestra.run.smithi001.stdout:2869 punch from 0x4e551ed to 0x4e5a624, (0x5437 bytes) 2024-09-13T04:58:55.245 INFO:teuthology.orchestra.run.smithi001.stdout:2871 read 0x2818c0 thru 0x28b585 (0x9cc6 bytes) 2024-09-13T04:58:55.245 INFO:teuthology.orchestra.run.smithi001.stdout:2873 writesame 0x4a940c7 thru 0x4aa186e data_size 0x8(0xd7a8 bytes) 2024-09-13T04:58:55.248 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:55.247+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:55.253 INFO:teuthology.orchestra.run.smithi001.stdout:2874 punch from 0xc381ff to 0xc3b8fc, (0x36fd bytes) 2024-09-13T04:58:55.253 INFO:teuthology.orchestra.run.smithi001.stdout:2876 write 0x6210 thru 0xf20d (0x8ffe bytes) 2024-09-13T04:58:55.254 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:55.254 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:55.256 INFO:teuthology.orchestra.run.smithi001.stdout:2877 writesame 0x4f01c3 thru 0x500054 data_size 0x46(0xfe92 bytes) 2024-09-13T04:58:55.259 INFO:teuthology.orchestra.run.smithi001.stdout:2878 read 0x837eac4 thru 0x838ba4d (0xcf8a bytes) 2024-09-13T04:58:55.260 INFO:teuthology.orchestra.run.smithi001.stdout:2883 writesame 0xdd7b37 thru 0xdda6cc data_size 0x7(0x2b96 bytes) 2024-09-13T04:58:55.263 INFO:teuthology.orchestra.run.smithi001.stdout:2884 trunc from 0x97788a6 to 0x9189735 2024-09-13T04:58:55.271 INFO:teuthology.orchestra.run.smithi001.stdout:2885 read 0x38f74e thru 0x39c80a (0xd0bd bytes) 2024-09-13T04:58:55.273 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:55.273 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:2886 write 0xc4f2734 thru 0xc4fd1e2 (0xaaaf bytes) 2024-09-13T04:58:55.287 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.288 INFO:teuthology.orchestra.run.smithi001.stdout:2889 write 0xb77ce6e thru 0xb7835b6 (0x6749 bytes) 2024-09-13T04:58:55.294 INFO:teuthology.orchestra.run.smithi001.stdout:2890 writesame 0x34b3095 thru 0x34b560e data_size 0x29(0x257a bytes) 2024-09-13T04:58:55.296 INFO:teuthology.orchestra.run.smithi001.stdout:2891 write 0x468a9b thru 0x46a72a (0x1c90 bytes) 2024-09-13T04:58:55.307 INFO:teuthology.orchestra.run.smithi001.stdout:2892 write 0xe5b5c34 thru 0xe5b81bf (0x258c bytes) 2024-09-13T04:58:55.308 INFO:teuthology.orchestra.run.smithi001.stdout:2893 read 0xb8097e9 thru 0xb810020 (0x6838 bytes) 2024-09-13T04:58:55.309 INFO:teuthology.orchestra.run.smithi001.stdout:2894 punch from 0xc1dec24 to 0xc1e5f32, (0x730e bytes) 2024-09-13T04:58:55.309 INFO:teuthology.orchestra.run.smithi001.stdout:2895 writesame 0x92e82fe thru 0x92ed296 data_size 0x29(0x4f99 bytes) 2024-09-13T04:58:55.314 INFO:teuthology.orchestra.run.smithi001.stdout:2897 read 0x717cc28 thru 0x718a46b (0xd844 bytes) 2024-09-13T04:58:55.314 INFO:teuthology.orchestra.run.smithi001.stdout:2898 writesame 0xa28540b thru 0xa28cc41 data_size 0x5(0x7837 bytes) 2024-09-13T04:58:55.318 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:55.315+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:55.320 INFO:teuthology.orchestra.run.smithi001.stdout:2899 read 0x5ef3dd2 thru 0x5f00365 (0xc594 bytes) 2024-09-13T04:58:55.320 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.321 INFO:teuthology.orchestra.run.smithi001.stdout:2901 writesame 0x226ce49 thru 0x2277730 data_size 0x2f(0xa8e8 bytes) 2024-09-13T04:58:55.325 INFO:teuthology.orchestra.run.smithi001.stdout:2903 read 0x1e7a394 thru 0x1e87c3b (0xd8a8 bytes) 2024-09-13T04:58:55.326 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.326 INFO:teuthology.orchestra.run.smithi001.stdout:2906 punch from 0xab27b3e to 0xab340ce, (0xc590 bytes) 2024-09-13T04:58:55.326 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.334 INFO:teuthology.orchestra.run.smithi001.stdout:2908 trunc from 0xe5b81c0 to 0x78982a7 2024-09-13T04:58:55.339 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:55.339 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:55.347 INFO:teuthology.orchestra.run.smithi001.stdout:2909 read 0x188583b thru 0x18889ae (0x3174 bytes) 2024-09-13T04:58:55.348 INFO:teuthology.orchestra.run.smithi001.stdout:2910 read 0x457c700 thru 0x458322c (0x6b2d bytes) 2024-09-13T04:58:55.348 INFO:teuthology.orchestra.run.smithi001.stdout:2911 write 0x2e7d645 thru 0x2e86321 (0x8cdd bytes) 2024-09-13T04:58:55.355 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:55.365 INFO:teuthology.orchestra.run.smithi001.stdout:2912 write 0xbb4cade thru 0xbb53cab (0x71ce bytes) 2024-09-13T04:58:55.369 INFO:teuthology.orchestra.run.smithi001.stdout:2913 write 0x60d2ff7 thru 0x60d68cf (0x38d9 bytes) 2024-09-13T04:58:55.373 INFO:teuthology.orchestra.run.smithi001.stdout:2914 write 0x1c426e3 thru 0x1c443ea (0x1d08 bytes) 2024-09-13T04:58:55.378 INFO:teuthology.orchestra.run.smithi001.stdout:2915 trunc from 0xbb53cac to 0xd77dacd 2024-09-13T04:58:55.385 INFO:teuthology.orchestra.run.smithi001.stdout:2918 punch from 0x96bf01a to 0x96c396a, (0x4950 bytes) 2024-09-13T04:58:55.385 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.399 INFO:teuthology.orchestra.run.smithi001.stdout:2921 trunc from 0xd77dacd to 0x1b40073 2024-09-13T04:58:55.436 INFO:teuthology.orchestra.run.smithi001.stdout:2922 write 0x7f40edc thru 0x7f44b6d (0x3c92 bytes) 2024-09-13T04:58:55.442 INFO:teuthology.orchestra.run.smithi001.stdout:2923 writesame 0x15f40f8 thru 0x15fba52 data_size 0x1(0x795b bytes) 2024-09-13T04:58:55.444 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:55.443+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:55.482 INFO:teuthology.orchestra.run.smithi001.stdout:2924 write 0x1504aba thru 0x150f3b3 (0xa8fa bytes) 2024-09-13T04:58:55.487 INFO:teuthology.orchestra.run.smithi001.stdout:2925 write 0x462a2c thru 0x470fb0 (0xe585 bytes) 2024-09-13T04:58:55.500 INFO:teuthology.orchestra.run.smithi001.stdout:2926 trunc from 0x7f44b6e to 0x5bfd882 2024-09-13T04:58:55.520 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:55.515+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:55.531 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:55.532 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:55.546 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:58:55.556 INFO:teuthology.orchestra.run.smithi001.stdout:2927 writesame 0xcff3be2 thru 0xcff7bb0 data_size 0x63(0x3fcf bytes) 2024-09-13T04:58:55.556 INFO:teuthology.orchestra.run.smithi001.stdout:2928 punch from 0x314d394 to 0x3156896, (0x9502 bytes) 2024-09-13T04:58:55.573 INFO:teuthology.orchestra.run.smithi001.stdout:2929 trunc from 0xcff7bb1 to 0x5cd9859 2024-09-13T04:58:55.586 INFO:teuthology.orchestra.run.smithi001.stdout:2930 writesame 0x2b4dda8 thru 0x2b5d9e4 data_size 0x1(0xfc3d bytes) 2024-09-13T04:58:55.633 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:55.631+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:55.691 INFO:teuthology.orchestra.run.smithi001.stdout:2932 write 0x81c1431 thru 0x81cd25f (0xbe2f bytes) 2024-09-13T04:58:55.701 INFO:teuthology.orchestra.run.smithi001.stdout:2933 punch from 0x43ab103 to 0x43b9fb5, (0xeeb2 bytes) 2024-09-13T04:58:55.701 INFO:teuthology.orchestra.run.smithi001.stdout:2934 punch from 0x4254478 to 0x425b667, (0x71ef bytes) 2024-09-13T04:58:55.701 INFO:teuthology.orchestra.run.smithi001.stdout:2935 punch from 0x6c4e123 to 0x6c50901, (0x27de bytes) 2024-09-13T04:58:55.713 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:55.713 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:55.719 INFO:teuthology.orchestra.run.smithi001.stdout:2936 write 0x9988dda thru 0x9998892 (0xfab9 bytes) 2024-09-13T04:58:55.721 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:55.719+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:55.728 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:58:55.753 INFO:teuthology.orchestra.run.smithi001.stdout:2937 write 0xe5fd354 thru 0xe602fa7 (0x5c54 bytes) 2024-09-13T04:58:55.763 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.763 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:55.763 INFO:teuthology.orchestra.run.smithi001.stdout:2942 punch from 0x18b58dd to 0x18bf0e6, (0x9809 bytes) 2024-09-13T04:58:55.763 INFO:teuthology.orchestra.run.smithi001.stdout:2943 write 0x73d5fe9 thru 0x73e1838 (0xb850 bytes) 2024-09-13T04:58:55.774 INFO:teuthology.orchestra.run.smithi001.stdout:2944 punch from 0x6422871 to 0x642d88f, (0xb01e bytes) 2024-09-13T04:58:55.774 INFO:teuthology.orchestra.run.smithi001.stdout:2945 write 0x81e9532 thru 0x81f8de2 (0xf8b1 bytes) 2024-09-13T04:58:55.778 INFO:teuthology.orchestra.run.smithi001.stdout:2946 read 0xa949fb8 thru 0xa955802 (0xb84b bytes) 2024-09-13T04:58:55.779 INFO:teuthology.orchestra.run.smithi001.stdout:2947 writesame 0x6878678 thru 0x6878a69 data_size 0x5(0x3f2 bytes) 2024-09-13T04:58:55.788 INFO:teuthology.orchestra.run.smithi001.stdout:2948 punch from 0xc97b738 to 0xc9846c6, (0x8f8e bytes) 2024-09-13T04:58:55.789 INFO:teuthology.orchestra.run.smithi001.stdout:2949 write 0x58a3b43 thru 0x58b2070 (0xe52e bytes) 2024-09-13T04:58:55.800 INFO:teuthology.orchestra.run.smithi001.stdout:2950 read 0x7127b85 thru 0x71352c5 (0xd741 bytes) 2024-09-13T04:58:55.801 INFO:teuthology.orchestra.run.smithi001.stdout:2951 read 0x3b71b51 thru 0x3b75f1c (0x43cc bytes) 2024-09-13T04:58:55.801 INFO:teuthology.orchestra.run.smithi001.stdout:2953 clone 20 order 18 su 32768 sc 13 2024-09-13T04:58:55.823 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:55.819+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:55.898 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:55.899 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:55.913 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:58:55.915 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T04:58:55.923 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:55.919+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:55.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T04:58:56.024 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:56.023+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:56.030 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:56.030 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:56.044 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:58:56.132 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:56.131+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:56.172 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-13T04:58:56.215 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:56.215 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:56.227 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:56.223+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:56.229 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:58:56.328 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:56.323+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:56.400 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:56.400 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:56.415 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:58:56.461 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:56.459+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:56.518 INFO:teuthology.orchestra.run.smithi001.stdout:2955 write 0x1166b3b thru 0x117496e (0xde34 bytes) 2024-09-13T04:58:56.529 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:56.527+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:2956 write 0xcdb4021 thru 0xcdbed2e (0xad0e bytes) 2024-09-13T04:58:56.546 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:56.546 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:56.556 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.564 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:58:56.576 INFO:teuthology.orchestra.run.smithi001.stdout:2959 trunc from 0xe602fa8 to 0x4e5ee5 2024-09-13T04:58:56.631 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:56.627+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:56.666 INFO:teuthology.orchestra.run.smithi001.stdout:2960 write 0x5871ff1 thru 0x587478b (0x279b bytes) 2024-09-13T04:58:56.674 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.674 INFO:teuthology.orchestra.run.smithi001.stdout:2963 punch from 0x3db46f to 0x3e8946, (0xd4d7 bytes) 2024-09-13T04:58:56.683 INFO:teuthology.orchestra.run.smithi001.stdout:2964 trunc from 0x587478c to 0x996978b 2024-09-13T04:58:56.709 INFO:teuthology.orchestra.run.smithi001.stdout:2965 write 0xc887a17 thru 0xc89552a (0xdb14 bytes) 2024-09-13T04:58:56.715 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:56.715 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:56.721 INFO:teuthology.orchestra.run.smithi001.stdout:2966 writesame 0xa25fbae thru 0xa264d86 data_size 0x1(0x51d9 bytes) 2024-09-13T04:58:56.732 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:58:56.733 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:56.727+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:56.754 INFO:teuthology.orchestra.run.smithi001.stdout:2967 writesame 0x3fd9053 thru 0x3fdac1b data_size 0x1(0x1bc9 bytes) 2024-09-13T04:58:56.780 INFO:teuthology.orchestra.run.smithi001.stdout:2968 trunc from 0xc89552b to 0x3b14e4f 2024-09-13T04:58:56.800 INFO:teuthology.orchestra.run.smithi001.stdout:2969 punch from 0x26cdd60 to 0x26d6ae6, (0x8d86 bytes) 2024-09-13T04:58:56.801 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.813 INFO:teuthology.orchestra.run.smithi001.stdout:2971 writesame 0x5d87f44 thru 0x5d893c4 data_size 0x1(0x1481 bytes) 2024-09-13T04:58:56.824 INFO:teuthology.orchestra.run.smithi001.stdout:2973 writesame 0x6d9afd9 thru 0x6daaa54 data_size 0x5c(0xfa7c bytes) 2024-09-13T04:58:56.824 INFO:teuthology.orchestra.run.smithi001.stdout:2974 punch from 0x360f96e to 0x361b797, (0xbe29 bytes) 2024-09-13T04:58:56.824 INFO:teuthology.orchestra.run.smithi001.stdout:2976 read 0x62f046 thru 0x631ee3 (0x2e9e bytes) 2024-09-13T04:58:56.825 INFO:teuthology.orchestra.run.smithi001.stdout:2978 read 0x5bd7540 thru 0x5be018c (0x8c4d bytes) 2024-09-13T04:58:56.825 INFO:teuthology.orchestra.run.smithi001.stdout:2979 read 0x6ca050e thru 0x6ca4fea (0x4add bytes) 2024-09-13T04:58:56.825 INFO:teuthology.orchestra.run.smithi001.stdout:2980 punch from 0x35ade1d to 0x35b35d8, (0x57bb bytes) 2024-09-13T04:58:56.825 INFO:teuthology.orchestra.run.smithi001.stdout:2981 read 0x67034d9 thru 0x6703869 (0x391 bytes) 2024-09-13T04:58:56.825 INFO:teuthology.orchestra.run.smithi001.stdout:2982 read 0x5c150e4 thru 0x5c18216 (0x3133 bytes) 2024-09-13T04:58:56.825 INFO:teuthology.orchestra.run.smithi001.stdout:2984 read 0xd52379 thru 0xd62150 (0xfdd8 bytes) 2024-09-13T04:58:56.825 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.826 INFO:teuthology.orchestra.run.smithi001.stdout:2986 read 0x31ee3db thru 0x31f34bc (0x50e2 bytes) 2024-09-13T04:58:56.826 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.826 INFO:teuthology.orchestra.run.smithi001.stdout:2990 read 0x611e02a thru 0x6121a7e (0x3a55 bytes) 2024-09-13T04:58:56.826 INFO:teuthology.orchestra.run.smithi001.stdout:2991 read 0x2f92855 thru 0x2f9ce0a (0xa5b6 bytes) 2024-09-13T04:58:56.826 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.833 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:56.831+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:56.835 INFO:teuthology.orchestra.run.smithi001.stdout:2993 write 0x7943eee thru 0x794acb6 (0x6dc9 bytes) 2024-09-13T04:58:56.845 INFO:teuthology.orchestra.run.smithi001.stdout:2994 punch from 0x1995bcb to 0x199ee65, (0x929a bytes) 2024-09-13T04:58:56.845 INFO:teuthology.orchestra.run.smithi001.stdout:2995 read 0x44e4682 thru 0x44e80ef (0x3a6e bytes) 2024-09-13T04:58:56.845 INFO:teuthology.orchestra.run.smithi001.stdout:2996 punch from 0x649b481 to 0x64a27e3, (0x7362 bytes) 2024-09-13T04:58:56.866 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:56.866 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:56.869 INFO:teuthology.orchestra.run.smithi001.stdout:2997 write 0xd984dcb thru 0xd98a484 (0x56ba bytes) 2024-09-13T04:58:56.879 INFO:teuthology.orchestra.run.smithi001.stdout:2999 write 0x9cb5c0f thru 0x9cb7b1e (0x1f10 bytes) 2024-09-13T04:58:56.882 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:58:56.885 INFO:teuthology.orchestra.run.smithi001.stdout:3000 read 0x64e5375 thru 0x64f1e49 (0xcad5 bytes) 2024-09-13T04:58:56.886 INFO:teuthology.orchestra.run.smithi001.stdout:3001 punch from 0x267a2fd to 0x2686793, (0xc496 bytes) 2024-09-13T04:58:56.902 INFO:teuthology.orchestra.run.smithi001.stdout:3002 trunc from 0xd98a485 to 0x64e0723 2024-09-13T04:58:56.922 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T04:58:56.925 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.925 INFO:teuthology.orchestra.run.smithi001.stdout:3005 punch from 0x3662455 to 0x3670cd6, (0xe881 bytes) 2024-09-13T04:58:56.925 INFO:teuthology.orchestra.run.smithi001.stdout:3007 read 0x555a3c1 thru 0x5569021 (0xec61 bytes) 2024-09-13T04:58:56.926 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.933 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:56.927+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:56.940 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:58:56.943 INFO:teuthology.orchestra.run.smithi001.stdout:3011 writesame 0x99145e8 thru 0x9921329 data_size 0x56(0xcd42 bytes) 2024-09-13T04:58:56.944 INFO:teuthology.orchestra.run.smithi001.stdout:3012 read 0x385e696 thru 0x3864f77 (0x68e2 bytes) 2024-09-13T04:58:56.944 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.955 INFO:teuthology.orchestra.run.smithi001.stdout:3015 write 0xae39150 thru 0xae47c6b (0xeb1c bytes) 2024-09-13T04:58:56.956 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T04:58:56.963 INFO:teuthology.orchestra.run.smithi001.stdout:3016 write 0x2e0e86d thru 0x2e0f4bf (0xc53 bytes) 2024-09-13T04:58:56.971 INFO:teuthology.orchestra.run.smithi001.stdout:3017 read 0x5bbb7bd thru 0x5bc78c7 (0xc10b bytes) 2024-09-13T04:58:56.971 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.971 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.971 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:56.971 INFO:teuthology.orchestra.run.smithi001.stdout:3022 write 0x502cc3d thru 0x502d690 (0xa54 bytes) 2024-09-13T04:58:56.990 INFO:teuthology.orchestra.run.smithi001.stdout:3023 trunc from 0xae47c6c to 0x5de247f 2024-09-13T04:58:57.007 INFO:teuthology.orchestra.run.smithi001.stdout:3028 read 0x110faf5 thru 0x111a389 (0xa895 bytes) 2024-09-13T04:58:57.025 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:57.025 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:57.029 INFO:teuthology.orchestra.run.smithi001.stdout:3030 write 0xe6767ab thru 0xe67b8eb (0x5141 bytes) 2024-09-13T04:58:57.034 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:57.031+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:57.039 INFO:teuthology.orchestra.run.smithi001.stdout:3032 write 0xa7a9e17 thru 0xa7aab62 (0xd4c bytes) 2024-09-13T04:58:57.044 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:58:57.045 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.046 INFO:teuthology.orchestra.run.smithi001.stdout:3034 writesame 0x9888e27 thru 0x9894d00 data_size 0x6(0xbeda bytes) 2024-09-13T04:58:57.056 INFO:teuthology.orchestra.run.smithi001.stdout:3036 write 0xae65143 thru 0xae72eb7 (0xdd75 bytes) 2024-09-13T04:58:57.073 INFO:teuthology.orchestra.run.smithi001.stdout:3037 trunc from 0xe67b8ec to 0xce4d289 2024-09-13T04:58:57.101 INFO:teuthology.orchestra.run.smithi001.stdout:3038 writesame 0xd7a1fc6 thru 0xd7a7789 data_size 0x52(0x57c4 bytes) 2024-09-13T04:58:57.101 INFO:teuthology.orchestra.run.smithi001.stdout:3039 write 0x91931eb thru 0x919378a (0x5a0 bytes) 2024-09-13T04:58:57.107 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.107 INFO:teuthology.orchestra.run.smithi001.stdout:3041 writesame 0x3c55295 thru 0x3c575d0 data_size 0xb(0x233c bytes) 2024-09-13T04:58:57.114 INFO:teuthology.orchestra.run.smithi001.stdout:3042 read 0x1abe850 thru 0x1ac2104 (0x38b5 bytes) 2024-09-13T04:58:57.114 INFO:teuthology.orchestra.run.smithi001.stdout:3044 punch from 0x8b24adc to 0x8b3353d, (0xea61 bytes) 2024-09-13T04:58:57.114 INFO:teuthology.orchestra.run.smithi001.stdout:3048 write 0x9d52909 thru 0x9d6008f (0xd787 bytes) 2024-09-13T04:58:57.130 INFO:teuthology.orchestra.run.smithi001.stdout:3049 writesame 0x299da95 thru 0x29aac92 data_size 0x1(0xd1fe bytes) 2024-09-13T04:58:57.135 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:57.131+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:57.162 INFO:teuthology.orchestra.run.smithi001.stdout:3050 punch from 0x2d00b76 to 0x2d04e05, (0x428f bytes) 2024-09-13T04:58:57.162 INFO:teuthology.orchestra.run.smithi001.stdout:3054 write 0x152267a thru 0x152473d (0x20c4 bytes) 2024-09-13T04:58:57.169 INFO:teuthology.orchestra.run.smithi001.stdout:3056 write 0x2027886 thru 0x2036fa2 (0xf71d bytes) 2024-09-13T04:58:57.189 INFO:teuthology.orchestra.run.smithi001.stdout:3058 trunc from 0xd7a778a to 0xb1d161a 2024-09-13T04:58:57.198 INFO:teuthology.orchestra.run.smithi001.stdout:3059 read 0x98a15f9 thru 0x98b15da (0xffe2 bytes) 2024-09-13T04:58:57.205 INFO:teuthology.orchestra.run.smithi001.stdout:3060 trunc from 0xb1d161a to 0x62c5b02 2024-09-13T04:58:57.216 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:57.216 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:57.225 INFO:teuthology.orchestra.run.smithi001.stdout:3061 read 0x5c709cb thru 0x5c73e0e (0x3444 bytes) 2024-09-13T04:58:57.236 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:57.235+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:57.240 INFO:teuthology.orchestra.run.smithi001.stdout:3063 writesame 0xb0b7dd1 thru 0xb0bea6f data_size 0x45(0x6c9f bytes) 2024-09-13T04:58:57.242 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:58:57.250 INFO:teuthology.orchestra.run.smithi001.stdout:3064 write 0xc65885a thru 0xc65de62 (0x5609 bytes) 2024-09-13T04:58:57.257 INFO:teuthology.orchestra.run.smithi001.stdout:3065 punch from 0xb3ff8d2 to 0xb4081df, (0x890d bytes) 2024-09-13T04:58:57.257 INFO:teuthology.orchestra.run.smithi001.stdout:3066 write 0xb64d843 thru 0xb65bb17 (0xe2d5 bytes) 2024-09-13T04:58:57.269 INFO:teuthology.orchestra.run.smithi001.stdout:3069 trunc from 0xc65de63 to 0xcab5c4c 2024-09-13T04:58:57.277 INFO:teuthology.orchestra.run.smithi001.stdout:3070 write 0x5c42e40 thru 0x5c450fc (0x22bd bytes) 2024-09-13T04:58:57.285 INFO:teuthology.orchestra.run.smithi001.stdout:3072 punch from 0xc035039 to 0xc036641, (0x1608 bytes) 2024-09-13T04:58:57.285 INFO:teuthology.orchestra.run.smithi001.stdout:3074 read 0xc305f04 thru 0xc306b5f (0xc5c bytes) 2024-09-13T04:58:57.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:58:57.302 INFO:teuthology.orchestra.run.smithi001.stdout:3075 trunc from 0xcab5c4c to 0x1374ee2 2024-09-13T04:58:57.331 INFO:teuthology.orchestra.run.smithi001.stdout:3077 trunc from 0x1374ee2 to 0x5b75d62 2024-09-13T04:58:57.337 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:57.335+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:57.339 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.339 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.339 INFO:teuthology.orchestra.run.smithi001.stdout:3083 read 0x3e5c836 thru 0x3e699e8 (0xd1b3 bytes) 2024-09-13T04:58:57.339 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.339 INFO:teuthology.orchestra.run.smithi001.stdout:3085 punch from 0x142a347 to 0x142c23d, (0x1ef6 bytes) 2024-09-13T04:58:57.340 INFO:teuthology.orchestra.run.smithi001.stdout:3086 read 0x31793bf thru 0x3185f30 (0xcb72 bytes) 2024-09-13T04:58:57.359 INFO:teuthology.orchestra.run.smithi001.stdout:3087 writesame 0xe0da8dd thru 0xe0e0213 data_size 0x3(0x5937 bytes) 2024-09-13T04:58:57.359 INFO:teuthology.orchestra.run.smithi001.stdout:3089 read 0xb13f479 thru 0xb14412d (0x4cb5 bytes) 2024-09-13T04:58:57.359 INFO:teuthology.orchestra.run.smithi001.stdout:3090 read 0x32016da thru 0x320a55c (0x8e83 bytes) 2024-09-13T04:58:57.360 INFO:teuthology.orchestra.run.smithi001.stdout:3091 write 0x6c7c1ab thru 0x6c85eac (0x9d02 bytes) 2024-09-13T04:58:57.370 INFO:teuthology.orchestra.run.smithi001.stdout:3092 punch from 0xa7f714 to 0xa81fde, (0x28ca bytes) 2024-09-13T04:58:57.370 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.370 INFO:teuthology.orchestra.run.smithi001.stdout:3095 read 0xe003856 thru 0xe00dab0 (0xa25b bytes) 2024-09-13T04:58:57.370 INFO:teuthology.orchestra.run.smithi001.stdout:3096 read 0x6825069 thru 0x682deed (0x8e85 bytes) 2024-09-13T04:58:57.371 INFO:teuthology.orchestra.run.smithi001.stdout:3099 read 0x88e26dd thru 0x88e4956 (0x227a bytes) 2024-09-13T04:58:57.371 INFO:teuthology.orchestra.run.smithi001.stdout:3100 punch from 0x79a3c1 to 0x7a6fa0, (0xcbdf bytes) 2024-09-13T04:58:57.371 INFO:teuthology.orchestra.run.smithi001.stdout:3101 punch from 0x7fefee2 to 0x7ff44c7, (0x45e5 bytes) 2024-09-13T04:58:57.371 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.371 INFO:teuthology.orchestra.run.smithi001.stdout:3103 read 0x68973f7 thru 0x689b5da (0x41e4 bytes) 2024-09-13T04:58:57.371 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.371 INFO:teuthology.orchestra.run.smithi001.stdout:3105 read 0x4d77bbf thru 0x4d7b830 (0x3c72 bytes) 2024-09-13T04:58:57.371 INFO:teuthology.orchestra.run.smithi001.stdout:3107 writesame 0x317e533 thru 0x318989a data_size 0x4(0xb368 bytes) 2024-09-13T04:58:57.386 INFO:teuthology.orchestra.run.smithi001.stdout:3108 write 0x6803755 thru 0x6804138 (0x9e4 bytes) 2024-09-13T04:58:57.387 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:58:57.387 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:58:57.400 INFO:teuthology.orchestra.run.smithi001.stdout:3109 write 0xe5d45b5 thru 0xe5df056 (0xaaa2 bytes) 2024-09-13T04:58:57.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:58:57.408 INFO:teuthology.orchestra.run.smithi001.stdout:3112 writesame 0x19c29df thru 0x19ca801 data_size 0x7(0x7e23 bytes) 2024-09-13T04:58:57.418 INFO:teuthology.orchestra.run.smithi001.stdout:3113 write 0x64f109c thru 0x64f9167 (0x80cc bytes) 2024-09-13T04:58:57.434 INFO:teuthology.orchestra.run.smithi001.stdout:3114 trunc from 0xe5df057 to 0xa66ddbe 2024-09-13T04:58:57.438 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:57.435+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:57.443 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:57.444 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:57.453 INFO:teuthology.orchestra.run.smithi001.stdout:3115 punch from 0x99237fa to 0x9932046, (0xe84c bytes) 2024-09-13T04:58:57.453 INFO:teuthology.orchestra.run.smithi001.stdout:3116 read 0x8a9e785 thru 0x8a9fc5d (0x14d9 bytes) 2024-09-13T04:58:57.458 INFO:teuthology.orchestra.run.smithi001.stdout:3118 trunc from 0xa66ddbe to 0x61a3065 2024-09-13T04:58:57.462 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:57.481 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:57.491 INFO:teuthology.orchestra.run.smithi001.stdout:3119 writesame 0xbf51d10 thru 0xbf6158a data_size 0x1(0xf87b bytes) 2024-09-13T04:58:57.491 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.492 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.492 INFO:teuthology.orchestra.run.smithi001.stdout:3123 punch from 0x1864cb0 to 0x186dd6c, (0x90bc bytes) 2024-09-13T04:58:57.500 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:57.500 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:57.502 INFO:teuthology.orchestra.run.smithi001.stdout:3124 write 0xecbc7ac thru 0xecc3cae (0x7503 bytes) 2024-09-13T04:58:57.512 INFO:teuthology.orchestra.run.smithi001.stdout:3125 read 0x6719764 thru 0x6721205 (0x7aa2 bytes) 2024-09-13T04:58:57.512 INFO:teuthology.orchestra.run.smithi001.stdout:3126 write 0x446e4ee thru 0x446e566 (0x79 bytes) 2024-09-13T04:58:57.518 INFO:teuthology.orchestra.run.smithi001.stdout:3127 write 0x6c1f4b2 thru 0x6c2490e (0x545d bytes) 2024-09-13T04:58:57.522 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:57.522 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:57.527 INFO:teuthology.orchestra.run.smithi001.stdout:3128 write 0x16a5f3f thru 0x16a8609 (0x26cb bytes) 2024-09-13T04:58:57.528 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:57.528 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:57.528 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:57.528 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:57.536 INFO:teuthology.orchestra.run.smithi001.stdout:3130 read 0x48f7f6e thru 0x4905574 (0xd607 bytes) 2024-09-13T04:58:57.536 INFO:teuthology.orchestra.run.smithi001.stdout:3131 write 0x6cb86f3 thru 0x6cc0515 (0x7e23 bytes) 2024-09-13T04:58:57.539 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:57.535+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:57.541 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:58:57.544 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:57.545 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:57.548 INFO:teuthology.orchestra.run.smithi001.stdout:3132 write 0xd227514 thru 0xd227c00 (0x6ed bytes) 2024-09-13T04:58:57.556 INFO:teuthology.orchestra.run.smithi001.stdout:3133 trunc from 0xecc3caf to 0xce1b378 2024-09-13T04:58:57.567 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:57.572 INFO:teuthology.orchestra.run.smithi001.stdout:3134 write 0x69c9fd5 thru 0x69cf8b3 (0x58df bytes) 2024-09-13T04:58:57.583 INFO:teuthology.orchestra.run.smithi001.stdout:3135 trunc from 0xce1b378 to 0x93eeb75 2024-09-13T04:58:57.589 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:57.589 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:57.591 INFO:teuthology.orchestra.run.smithi001.stdout:3136 read 0x1646f3 thru 0x17011b (0xba29 bytes) 2024-09-13T04:58:57.594 INFO:teuthology.orchestra.run.smithi001.stdout:3137 write 0x8ddc1bb thru 0x8de8cf4 (0xcb3a bytes) 2024-09-13T04:58:57.607 INFO:teuthology.orchestra.run.smithi001.stdout:3138 read 0x87d43c5 thru 0x87df2e3 (0xaf1f bytes) 2024-09-13T04:58:57.609 INFO:teuthology.orchestra.run.smithi001.stdout:3139 trunc from 0x93eeb75 to 0x83e6180 2024-09-13T04:58:57.636 INFO:teuthology.orchestra.run.smithi001.stdout:3140 write 0xc0d3044 thru 0xc0e2312 (0xf2cf bytes) 2024-09-13T04:58:57.640 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:57.635+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:57.648 INFO:teuthology.orchestra.run.smithi001.stdout:3141 write 0x3569af0 thru 0x3577f2f (0xe440 bytes) 2024-09-13T04:58:57.657 INFO:teuthology.orchestra.run.smithi001.stdout:3142 writesame 0xbbbe6c9 thru 0xbbbeb20 data_size 0x1(0x458 bytes) 2024-09-13T04:58:57.663 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:57.663 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:57.663 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:57.663 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:57.663 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:57.665 INFO:teuthology.orchestra.run.smithi001.stdout:3143 read 0xa002192 thru 0xa00a3d7 (0x8246 bytes) 2024-09-13T04:58:57.672 INFO:teuthology.orchestra.run.smithi001.stdout:3144 write 0xc176e4d thru 0xc17f003 (0x81b7 bytes) 2024-09-13T04:58:57.677 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:58:57.690 INFO:teuthology.orchestra.run.smithi001.stdout:3146 write 0xed17162 thru 0xed24c07 (0xdaa6 bytes) 2024-09-13T04:58:57.699 INFO:teuthology.orchestra.run.smithi001.stdout:3147 write 0x26bcc65 thru 0x26c91e4 (0xc580 bytes) 2024-09-13T04:58:57.715 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.716 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.716 INFO:teuthology.orchestra.run.smithi001.stdout:3152 read 0xc72c08c thru 0xc72ef5f (0x2ed4 bytes) 2024-09-13T04:58:57.716 INFO:teuthology.orchestra.run.smithi001.stdout:3153 read 0xde9afa5 thru 0xdea447d (0x94d9 bytes) 2024-09-13T04:58:57.716 INFO:teuthology.orchestra.run.smithi001.stdout:3155 read 0x4665eb6 thru 0x4666c5a (0xda5 bytes) 2024-09-13T04:58:57.716 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.716 INFO:teuthology.orchestra.run.smithi001.stdout:3157 write 0x3e31976 thru 0x3e40bc4 (0xf24f bytes) 2024-09-13T04:58:57.727 INFO:teuthology.orchestra.run.smithi001.stdout:3158 writesame 0x1d31e65 thru 0x1d3b6e4 data_size 0x7a(0x9880 bytes) 2024-09-13T04:58:57.736 INFO:teuthology.orchestra.run.smithi001.stdout:3159 writesame 0xde91244 thru 0xde9283d data_size 0x1d(0x15fa bytes) 2024-09-13T04:58:57.741 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:57.739+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:57.742 INFO:teuthology.orchestra.run.smithi001.stdout:3160 write 0x7f41a58 thru 0x7f44cc6 (0x326f bytes) 2024-09-13T04:58:57.752 INFO:teuthology.orchestra.run.smithi001.stdout:3161 trunc from 0xed24c08 to 0xa04d8d8 2024-09-13T04:58:57.771 INFO:teuthology.orchestra.run.smithi001.stdout:3163 write 0x9e4e5f7 thru 0x9e53fcf (0x59d9 bytes) 2024-09-13T04:58:57.787 INFO:teuthology.orchestra.run.smithi001.stdout:3166 write 0xc2135b6 thru 0xc21dbd9 (0xa624 bytes) 2024-09-13T04:58:57.797 INFO:teuthology.orchestra.run.smithi001.stdout:3167 read 0xbf99637 thru 0xbfa0488 (0x6e52 bytes) 2024-09-13T04:58:57.797 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:57.797 INFO:teuthology.orchestra.run.smithi001.stdout:3169 read 0xa816fae thru 0xa81cf16 (0x5f69 bytes) 2024-09-13T04:58:57.797 INFO:teuthology.orchestra.run.smithi001.stdout:3170 read 0x995c77a thru 0x996a32a (0xdbb1 bytes) 2024-09-13T04:58:57.797 INFO:teuthology.orchestra.run.smithi001.stdout:3171 punch from 0x423e3a6 to 0x424ab7b, (0xc7d5 bytes) 2024-09-13T04:58:57.797 INFO:teuthology.orchestra.run.smithi001.stdout:3173 write 0x12ddd9a thru 0x12e022c (0x2493 bytes) 2024-09-13T04:58:57.811 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:57.811 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:57.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:57.811 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:57.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.349037027999998, 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:37637 3.11 e41/24 3:88b77d11:::rbd_data.112a66af081a.0000000000000000:head v 41'72, mlcod=41'70)", 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.454838072000001, 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027255564999999999, 2024-09-13T04:58:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355356+0000", 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356532+0000", 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001176599 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356563+0000", 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0916e-05 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356576+0000", 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3293e-05 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356700+0000", 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123779 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356746+0000", 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6242000000000002e-05 2024-09-13T04:58:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382519+0000", 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025772856 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382614+0000", 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4315999999999996e-05 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39466 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.713890654, 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022036103000000001, 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39466 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096315+0000", 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096326+0000", 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1596999999999999e-05 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096332+0000", 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8560000000000003e-06 2024-09-13T04:58:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112144+0000", 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015812001999999999 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112220+0000", 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5354999999999999e-05 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.118342+0000", 2024-09-13T04:58:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061219409999999997 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40000 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 507904~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678931928000001, 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019326534999999999, 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40000 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131271+0000", 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9330000000000004e-06 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131278+0000", 2024-09-13T04:58:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9269999999999999e-06 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131282+0000", 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8729999999999998e-06 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150497+0000", 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019214709 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150537+0000", 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0197000000000002e-05 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150591+0000", 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3352e-05 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40001 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 638976~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678908842999999, 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019422915999999998, 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40001 2024-09-13T04:58:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131289+0000", 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.491e-06 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131306+0000", 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6489000000000001e-05 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131311+0000", 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150619+0000", 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019307642 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150657+0000", 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8015000000000001e-05 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150710+0000", 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3109999999999998e-05 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40004 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 770048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678880080999999, 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019514348000000001, 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40004 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131318+0000", 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8159999999999998e-06 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131325+0000", 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5370000000000003e-06 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131330+0000", 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6619999999999996e-06 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150741+0000", 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019410981000000001 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150778+0000", 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7793999999999998e-05 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150830+0000", 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1974000000000001e-05 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40006 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 901120~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678862166000002, 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019616892, 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40006 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131336+0000", 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1940000000000001e-06 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131342+0000", 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0599999999999996e-06 2024-09-13T04:58:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131346+0000", 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5569999999999997e-06 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150858+0000", 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019512057999999999 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150897+0000", 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8903999999999999e-05 2024-09-13T04:58:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150951+0000", 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3467999999999997e-05 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40008 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1032192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678845513999999, 2024-09-13T04:58:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019721992000000001, 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40008 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131350+0000", 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131360+0000", 2024-09-13T04:58:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5079999999999994e-06 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131367+0000", 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8210000000000004e-06 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131371+0000", 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8860000000000003e-06 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150979+0000", 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019607655000000002 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151020+0000", 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0921999999999999e-05 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151073+0000", 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2567e-05 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40010 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1163264~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678819909000001, 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019817818000000001, 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40010 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131378+0000", 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131384+0000", 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0780000000000001e-06 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131389+0000", 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5079999999999999e-06 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151101+0000", 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019712159 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151141+0000", 2024-09-13T04:58:57.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0679000000000001e-05 2024-09-13T04:58:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151194+0000", 2024-09-13T04:58:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2589000000000001e-05 2024-09-13T04:58:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40012 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1294336~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678803552000002, 2024-09-13T04:58:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019922019999999999, 2024-09-13T04:58:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40012 2024-09-13T04:58:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:58:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131392+0000", 2024-09-13T04:58:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131394+0000", 2024-09-13T04:58:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.029e-06 2024-09-13T04:58:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131400+0000", 2024-09-13T04:58:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0560000000000001e-06 2024-09-13T04:58:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131424+0000", 2024-09-13T04:58:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3288000000000001e-05 2024-09-13T04:58:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151223+0000", 2024-09-13T04:58:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019799231 2024-09-13T04:58:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151263+0000", 2024-09-13T04:58:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9920999999999999e-05 2024-09-13T04:58:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151315+0000", 2024-09-13T04:58:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1826000000000002e-05 2024-09-13T04:58:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40014 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1425408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678767015999998, 2024-09-13T04:58:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020004087, 2024-09-13T04:58:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40014 2024-09-13T04:58:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:58:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131432+0000", 2024-09-13T04:58:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9780000000000001e-06 2024-09-13T04:58:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131441+0000", 2024-09-13T04:58:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9590000000000001e-06 2024-09-13T04:58:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131447+0000", 2024-09-13T04:58:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9410000000000004e-06 2024-09-13T04:58:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151343+0000", 2024-09-13T04:58:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019896113999999999 2024-09-13T04:58:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151381+0000", 2024-09-13T04:58:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7969999999999997e-05 2024-09-13T04:58:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151433+0000", 2024-09-13T04:58:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2543000000000002e-05 2024-09-13T04:58:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40016 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1556480~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678714579000001, 2024-09-13T04:58:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020070607000000001, 2024-09-13T04:58:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40016 2024-09-13T04:58:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:58:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131481+0000", 2024-09-13T04:58:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131484+0000", 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.326e-06 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131492+0000", 2024-09-13T04:58:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0139999999999998e-06 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131505+0000", 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151461+0000", 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01995535 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151500+0000", 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8760999999999997e-05 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151552+0000", 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2533e-05 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40018 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1687552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678684754999999, 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020160419999999998, 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40018 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131514+0000", 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0740000000000001e-06 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131520+0000", 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4479999999999999e-06 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131526+0000", 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151580+0000", 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020054424000000001 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151619+0000", 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9229e-05 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151672+0000", 2024-09-13T04:58:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2493e-05 2024-09-13T04:58:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40019 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 114688~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.888 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678665781999999, 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020264445999999998, 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40019 2024-09-13T04:58:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131532+0000", 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.367e-06 2024-09-13T04:58:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131538+0000", 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1070000000000001e-06 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131543+0000", 2024-09-13T04:58:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3930000000000001e-06 2024-09-13T04:58:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151700+0000", 2024-09-13T04:58:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020157191000000001 2024-09-13T04:58:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151741+0000", 2024-09-13T04:58:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0864e-05 2024-09-13T04:58:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151795+0000", 2024-09-13T04:58:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3896e-05 2024-09-13T04:58:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40020 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678648881000001, 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020368440000000002, 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40020 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131549+0000", 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1110000000000002e-06 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131555+0000", 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3350000000000003e-06 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131560+0000", 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151823+0000", 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020262782999999999 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151863+0000", 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0151000000000002e-05 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151916+0000", 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2604e-05 2024-09-13T04:58:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40021 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1949696~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678632021999999, 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020470662000000001, 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40021 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131566+0000", 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.001e-06 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131571+0000", 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5110000000000003e-06 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131576+0000", 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151943+0000", 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020366649000000001 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151981+0000", 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8756e-05 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152035+0000", 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3335999999999999e-05 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40023 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.678609589000001, 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020567854, 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40023 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131589+0000", 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4140000000000001e-06 2024-09-13T04:58:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131597+0000", 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5220000000000007e-06 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131613+0000", 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5628999999999999e-05 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152063+0000", 2024-09-13T04:58:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020450032 2024-09-13T04:58:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152101+0000", 2024-09-13T04:58:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7886999999999998e-05 2024-09-13T04:58:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152154+0000", 2024-09-13T04:58:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3739999999999997e-05 2024-09-13T04:58:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40052 3.e 3:75db92e1:::rbd_header.112a6578f5c6:head [watch reconnect cookie 139644891391936 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.67857764, 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021419572000000001, 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40052 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131621+0000", 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9280000000000002e-06 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131635+0000", 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4467000000000001e-05 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131642+0000", 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.144e-06 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152171+0000", 2024-09-13T04:58:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020529544 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152220+0000", 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.153038+0000", 2024-09-13T04:58:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00081775299999999995 2024-09-13T04:58:57.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.919 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:49893 3.11 e48/24 3:88a8e927:::rbd_data.112ae3a54590.0000000000000005:head v 48'89, mlcod=47'87)", 2024-09-13T04:58:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.763606836000001, 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023644946999999999, 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:58:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048077+0000", 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001487817 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048104+0000", 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7056e-05 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048117+0000", 2024-09-13T04:58:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2914999999999999e-05 2024-09-13T04:58:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048200+0000", 2024-09-13T04:58:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3042999999999998e-05 2024-09-13T04:58:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048231+0000", 2024-09-13T04:58:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0834999999999999e-05 2024-09-13T04:58:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070186+0000", 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021955283999999999 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070234+0000", 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8325000000000002e-05 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:73381 3.3 e57/24 3:c56c84c9:::rbd_data.112aaf1f27a8.0000000000000000:head v 57'247, mlcod=57'245)", 2024-09-13T04:58:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:58:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.6028457849999995, 2024-09-13T04:58:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024031073999999999, 2024-09-13T04:58:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:58:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:58:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207348+0000", 2024-09-13T04:58:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:58:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209308+0000", 2024-09-13T04:58:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019604000000000002 2024-09-13T04:58:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209340+0000", 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1741e-05 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209355+0000", 2024-09-13T04:58:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5625999999999999e-05 2024-09-13T04:58:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209470+0000", 2024-09-13T04:58:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115047 2024-09-13T04:58:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209517+0000", 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6595999999999999e-05 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.231241+0000", 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021723592 2024-09-13T04:58:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.231381+0000", 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014075399999999999 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:3175 trunc from 0xc21dbda to 0x36b91eb 2024-09-13T04:58:57.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:58:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:3176 writesame 0x4ed0a6 thru 0x4f69d4 data_size 0x9b(0x992f bytes) 2024-09-13T04:58:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:3177 trunc from 0x36b91eb to 0x59f392e 2024-09-13T04:58:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:3178 punch from 0x48417d to 0x4872ed, (0x3170 bytes) 2024-09-13T04:58:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:3179 punch from 0x39a634f to 0x39b03be, (0xa06f bytes) 2024-09-13T04:58:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:3180 clone 21 order 20 su 262144 sc 4 2024-09-13T04:58:57.939 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:58:57.939 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T04:58:57.939 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:57.931+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:57.965 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T04:58:58.034 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone20 from 0x59f392e (overlap 0x4e5ee5) to 0x0 2024-09-13T04:58:58.037 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:58.031+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:58.104 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:58.104 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:58:58.105 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:58.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:58:58.139 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:58.135+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:58.168 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-13T04:58:58.239 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:58.235+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:58.290 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:58.291 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:58:58.291 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:58:58.291 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:58:58.291 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:58.324 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:58:58.366 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:58.363+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:58.441 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:58:58.439+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:58:58.494 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:58:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:58:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:58:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:58:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.328150464000004, 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:58:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:58:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.325709805000002, 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:58:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:58:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:58:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.036477511000001, 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:58:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:58:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.478877864000001, 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:58:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.464366812000002, 2024-09-13T04:58:58.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:58:58.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:58:58.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70501 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 901120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2253188179999999, 2024-09-13T04:58:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048712395999999998, 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70501 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266809+0000", 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266812+0000", 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3970000000000001e-06 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266816+0000", 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.729e-06 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266820+0000", 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0690000000000003e-06 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315355+0000", 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048534985000000003 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315417+0000", 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1765000000000002e-05 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315522+0000", 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105142 2024-09-13T04:58:58.531 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70502 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 905216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2252830820000007, 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048974827999999998, 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70502 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266845+0000", 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266851+0000", 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266857+0000", 2024-09-13T04:58:58.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7180000000000001e-06 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266861+0000", 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6569999999999996e-06 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315576+0000", 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048714396 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315687+0000", 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111118 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315821+0000", 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013375400000000001 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70503 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 860160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.536 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2252299480000008, 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049096338000000003, 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70503 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:58:58.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266898+0000", 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266901+0000", 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266906+0000", 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4600000000000002e-06 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266911+0000", 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7570000000000004e-06 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315859+0000", 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048947955000000001 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315920+0000", 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0906000000000003e-05 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315995+0000", 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5185000000000006e-05 2024-09-13T04:58:58.540 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70504 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 864256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2252143219999994, 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049340255, 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70504 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266917+0000", 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7180000000000001e-06 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266922+0000", 2024-09-13T04:58:58.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.028e-06 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266926+0000", 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4750000000000004e-06 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316035+0000", 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049109070999999997 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316136+0000", 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010061 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316255+0000", 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118884 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.545 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70505 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 868352~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2251989870000006, 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049486095000000001, 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70505 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266929+0000", 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266932+0000", 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.306e-06 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266936+0000", 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8230000000000002e-06 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266941+0000", 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.279e-06 2024-09-13T04:58:58.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316291+0000", 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049349968000000001 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316344+0000", 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3659999999999998e-05 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316416+0000", 2024-09-13T04:58:58.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1606999999999994e-05 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70506 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 856064~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2251845039999996, 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049736427, 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.550 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70506 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266946+0000", 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3240000000000001e-06 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266951+0000", 2024-09-13T04:58:58.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7489999999999998e-06 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266956+0000", 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9180000000000002e-06 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316460+0000", 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049503783000000003 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316555+0000", 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316681+0000", 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125513 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70507 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 872448~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2251543680000001, 2024-09-13T04:58:58.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049868537999999997, 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70507 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266977+0000", 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8459999999999999e-06 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266982+0000", 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3179999999999997e-06 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266986+0000", 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4320000000000001e-06 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316715+0000", 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049728886 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316772+0000", 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6805000000000002e-05 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316843+0000", 2024-09-13T04:58:58.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0928999999999998e-05 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70508 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 909312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2251271750000008, 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050098392999999998, 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70508 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:58:58.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267001+0000", 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267004+0000", 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2589999999999999e-06 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267009+0000", 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4410000000000001e-06 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267014+0000", 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6319999999999999e-06 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316884+0000", 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049869956 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316982+0000", 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8472000000000002e-05 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317100+0000", 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117977 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70509 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 876544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2251117639999993, 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050242336999999998, 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70509 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:58:58.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267019+0000", 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1969999999999999e-06 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267024+0000", 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267028+0000", 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2169999999999996e-06 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317133+0000", 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050105246999999999 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317190+0000", 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6623000000000003e-05 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317259+0000", 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9599999999999998e-05 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70510 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 913408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2250972099999995, 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050487833000000003, 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70510 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267031+0000", 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267033+0000", 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5660000000000001e-06 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267038+0000", 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7650000000000001e-06 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267042+0000", 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.233e-06 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317301+0000", 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050258540999999997 2024-09-13T04:58:58.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317403+0000", 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101727 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317519+0000", 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116685 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70511 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 884736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2250830500000003, 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050628583999999997, 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70511 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267045+0000", 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267048+0000", 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1430000000000001e-06 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267054+0000", 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7950000000000002e-06 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267059+0000", 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.318e-06 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317553+0000", 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050494202000000002 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317605+0000", 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2039000000000001e-05 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317674+0000", 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999996e-05 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70512 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 880640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2250312389999998, 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050833717, 2024-09-13T04:58:58.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70512 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267097+0000", 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267100+0000", 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7630000000000001e-06 2024-09-13T04:58:58.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267105+0000", 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2850000000000002e-06 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267109+0000", 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5070000000000002e-06 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317716+0000", 2024-09-13T04:58:58.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050606774 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317810+0000", 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3851000000000007e-05 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317931+0000", 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121177 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70513 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 892928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2250155360000008, 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050973768000000003, 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70513 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267115+0000", 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2139999999999999e-06 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267122+0000", 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9639999999999997e-06 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267127+0000", 2024-09-13T04:58:58.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4789999999999998e-06 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317967+0000", 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050840242000000001 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318019+0000", 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.189e-05 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318087+0000", 2024-09-13T04:58:58.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8312999999999995e-05 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70514 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2249984630000004, 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051225729999999997, 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70514 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267136+0000", 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1399999999999997e-06 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267141+0000", 2024-09-13T04:58:58.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0690000000000001e-06 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267145+0000", 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3830000000000002e-06 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318126+0000", 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050980481000000001 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318237+0000", 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110851 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318356+0000", 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119499 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70515 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 897024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:58:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.2249464280000009, 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051324634000000001, 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70515 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:58:58.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267184+0000", 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6529999999999998e-06 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267190+0000", 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267194+0000", 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4859999999999999e-06 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318381+0000", 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051186785999999998 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318436+0000", 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5180999999999999e-05 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318507+0000", 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0753999999999994e-05 2024-09-13T04:58:58.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:58:58.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:58:58.595 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:58:58.596 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:58:58.597 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:58.591+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:58.654 INFO:teuthology.orchestra.run.smithi001.stdout:3181 write 0xc4e4622 thru 0xc4ed130 (0x8b0f bytes) 2024-09-13T04:58:58.661 INFO:teuthology.orchestra.run.smithi001.stdout:3182 trunc from 0xc4ed131 to 0xcfd2c96 2024-09-13T04:58:58.673 INFO:teuthology.orchestra.run.smithi001.stdout:3183 trunc from 0xcfd2c96 to 0xee248f8 2024-09-13T04:58:58.684 INFO:teuthology.orchestra.run.smithi001.stdout:3185 read 0xcf08e78 thru 0xcf138fc (0xaa85 bytes) 2024-09-13T04:58:58.684 INFO:teuthology.orchestra.run.smithi001.stdout:3186 write 0x31be1c6 thru 0x31cbe7d (0xdcb8 bytes) 2024-09-13T04:58:58.697 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:58.691+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:58.701 INFO:teuthology.orchestra.run.smithi001.stdout:3187 write 0x9194e0f thru 0x919b171 (0x6363 bytes) 2024-09-13T04:58:58.711 INFO:teuthology.orchestra.run.smithi001.stdout:3188 read 0x727e20d thru 0x728cf74 (0xed68 bytes) 2024-09-13T04:58:58.711 INFO:teuthology.orchestra.run.smithi001.stdout:3190 write 0x8442dc9 thru 0x8447bb2 (0x4dea bytes) 2024-09-13T04:58:58.721 INFO:teuthology.orchestra.run.smithi001.stdout:3191 trunc from 0xee248f8 to 0xdc4ac90 2024-09-13T04:58:58.736 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:58.736 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:58.737 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:58.741 INFO:teuthology.orchestra.run.smithi001.stdout:3192 trunc from 0xdc4ac90 to 0xa449b1b 2024-09-13T04:58:58.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:58:58.762 INFO:teuthology.orchestra.run.smithi001.stdout:3193 writesame 0x124ca86 thru 0x12525d4 data_size 0x1(0x5b4f bytes) 2024-09-13T04:58:58.798 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:58.795+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:58.798 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:58.799 INFO:teuthology.orchestra.run.smithi001.stdout:3195 punch from 0x567c9c4 to 0x5683017, (0x6653 bytes) 2024-09-13T04:58:58.803 INFO:teuthology.orchestra.run.smithi001.stdout:3196 trunc from 0xa449b1b to 0xbf91015 2024-09-13T04:58:58.818 INFO:teuthology.orchestra.run.smithi001.stdout:3197 punch from 0xa08a3f9 to 0xa08f09e, (0x4ca5 bytes) 2024-09-13T04:58:58.819 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:58:58.819 INFO:teuthology.orchestra.run.smithi001.stdout:3199 punch from 0x3307f0f to 0x330f577, (0x7668 bytes) 2024-09-13T04:58:58.819 INFO:teuthology.orchestra.run.smithi001.stdout:3200 punch from 0x8011da8 to 0x801df21, (0xc179 bytes) 2024-09-13T04:58:58.836 INFO:teuthology.orchestra.run.smithi001.stdout:3202 write 0xd3c5e91 thru 0xd3d2d43 (0xceb3 bytes) 2024-09-13T04:58:58.845 INFO:teuthology.orchestra.run.smithi001.stdout:3203 write 0x5fdae4f thru 0x5fe8769 (0xd91b bytes) 2024-09-13T04:58:58.860 INFO:teuthology.orchestra.run.smithi001.stdout:3204 punch from 0x4689cff to 0x468c5c4, (0x28c5 bytes) 2024-09-13T04:58:58.860 INFO:teuthology.orchestra.run.smithi001.stdout:3206 punch from 0x51344bc to 0x513d592, (0x90d6 bytes) 2024-09-13T04:58:58.860 INFO:teuthology.orchestra.run.smithi001.stdout:3207 writesame 0x3472bb thru 0x34ccf6 data_size 0xf(0x5a3c bytes) 2024-09-13T04:58:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:58.878 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:58.878 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:58.878 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:58.892 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:58:58.892 INFO:teuthology.orchestra.run.smithi001.stdout:3208 write 0xa032e0b thru 0xa03b2d2 (0x84c8 bytes) 2024-09-13T04:58:58.899 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:58.895+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:58.904 INFO:teuthology.orchestra.run.smithi001.stdout:3209 writesame 0x7a6b638 thru 0x7a731c7 data_size 0x18(0x7b90 bytes) 2024-09-13T04:58:58.946 INFO:teuthology.orchestra.run.smithi001.stdout:3210 trunc from 0xd3d2d44 to 0xdcd60c 2024-09-13T04:58:58.949 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T04:58:58.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T04:58:58.979 INFO:teuthology.orchestra.run.smithi001.stdout:3212 read 0x2bd367 thru 0x2cafb6 (0xdc50 bytes) 2024-09-13T04:58:58.982 INFO:teuthology.orchestra.run.smithi001.stdout:3213 punch from 0x8103a to 0x8811b, (0x70e1 bytes) 2024-09-13T04:58:58.982 INFO:teuthology.orchestra.run.smithi001.stdout:3214 read 0x552e79 thru 0x55c4e8 (0x9670 bytes) 2024-09-13T04:58:58.982 INFO:teuthology.orchestra.run.smithi001.stdout:3215 punch from 0xc19664 to 0xc1abf2, (0x158e bytes) 2024-09-13T04:58:58.982 INFO:teuthology.orchestra.run.smithi001.stdout:3217 punch from 0x17b447 to 0x180e45, (0x59fe bytes) 2024-09-13T04:58:59.009 INFO:teuthology.orchestra.run.smithi001.stdout:3218 write 0xd6759ac thru 0xd67cee1 (0x7536 bytes) 2024-09-13T04:58:59.010 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:58:59.007+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:58:59.014 INFO:teuthology.orchestra.run.smithi001.stdout:3220 read 0x400b336 thru 0x40198a1 (0xe56c bytes) 2024-09-13T04:58:59.014 INFO:teuthology.orchestra.run.smithi001.stdout:3221 punch from 0x246d31c to 0x246d751, (0x435 bytes) 2024-09-13T04:58:59.015 INFO:teuthology.orchestra.run.smithi001.stdout:3222 writesame 0x1e328be thru 0x1e3fea1 data_size 0x1b(0xd5e4 bytes) 2024-09-13T04:58:59.019 INFO:teuthology.orchestra.run.smithi001.stdout:3223 write 0x56c277a thru 0x56c7c40 (0x54c7 bytes) 2024-09-13T04:58:59.024 INFO:teuthology.orchestra.run.smithi001.stdout:3224 read 0x2782d26 thru 0x2784ec3 (0x219e bytes) 2024-09-13T04:58:59.024 INFO:teuthology.orchestra.run.smithi001.stdout:3227 read 0xb98fb3a thru 0xb98fdf4 (0x2bb bytes) 2024-09-13T04:58:59.024 INFO:teuthology.orchestra.run.smithi001.stdout:3229 punch from 0xd48071b to 0xd48d251, (0xcb36 bytes) 2024-09-13T04:58:59.024 INFO:teuthology.orchestra.run.smithi001.stdout:3230 punch from 0x370e945 to 0x3716d44, (0x83ff bytes) 2024-09-13T04:58:59.024 INFO:teuthology.orchestra.run.smithi001.stdout:3231 punch from 0x78d2646 to 0x78d96c9, (0x7083 bytes) 2024-09-13T04:58:59.025 INFO:teuthology.orchestra.run.smithi001.stdout:3232 write 0x5ce1203 thru 0x5ceecc0 (0xdabe bytes) 2024-09-13T04:58:59.031 INFO:teuthology.orchestra.run.smithi001.stdout:3234 writesame 0x15b7f21 thru 0x15bf11c data_size 0xa(0x71fc bytes) 2024-09-13T04:58:59.038 INFO:teuthology.orchestra.run.smithi001.stdout:3235 punch from 0xc4f6b59 to 0xc501791, (0xac38 bytes) 2024-09-13T04:58:59.054 INFO:teuthology.orchestra.run.smithi001.stdout:3236 trunc from 0xd67cee2 to 0x26081f8 2024-09-13T04:58:59.069 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:59.069 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.2101785359999999, 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T04:58:59.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T04:58:59.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T04:58:59.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.210151625, 2024-09-13T04:58:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.2064059890000003, 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T04:58:59.079 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:58:59.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:58:59.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T04:58:59.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T04:58:59.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T04:58:59.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T04:58:59.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T04:58:59.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T04:58:59.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T04:58:59.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T04:58:59.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T04:58:59.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T04:58:59.086 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.086 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.087 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.087 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:58:59.087 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.2028626640000004, 2024-09-13T04:58:59.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T04:58:59.087 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T04:58:59.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T04:58:59.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T04:58:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T04:58:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T04:58:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T04:58:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T04:58:59.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T04:58:59.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T04:58:59.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T04:58:59.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.094 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.094 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.094 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:58:59.094 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.2028230359999998, 2024-09-13T04:58:59.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T04:58:59.095 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.095 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.095 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T04:58:59.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.096 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:58:59.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T04:58:59.099 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.100 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.100 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.100 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:58:59.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.2027993759999998, 2024-09-13T04:58:59.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T04:58:59.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.101 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T04:58:59.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.2027118269999999, 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.110 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.111 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:58:59.111 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.2026591709999996, 2024-09-13T04:58:59.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T04:58:59.111 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.111 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.112 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.112 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.112 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T04:58:59.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.112 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T04:58:59.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T04:58:59.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T04:58:59.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T04:58:59.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T04:58:59.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T04:58:59.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T04:58:59.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T04:58:59.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T04:58:59.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T04:58:59.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.118 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.118 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.118 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1987332799999999, 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T04:58:59.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:58:59.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:58:59.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T04:58:59.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T04:58:59.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T04:58:59.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T04:58:59.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T04:58:59.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T04:58:59.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:58:59.125 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1987076549999998, 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T04:58:59.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1986742609999999, 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.130 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:58:59.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.134 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1938843319999997, 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.135 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T04:58:59.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.139 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1904897029999999, 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:58:59.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1904170269999996, 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T04:58:59.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T04:58:59.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T04:58:59.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T04:58:59.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.148 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.186674666, 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.149 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.150 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.150 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.150 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.150 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T04:58:59.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.150 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:58:59.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:58:59.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T04:58:59.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T04:58:59.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T04:58:59.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T04:58:59.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T04:58:59.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T04:58:59.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T04:58:59.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T04:58:59.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T04:58:59.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T04:58:59.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T04:58:59.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T04:58:59.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T04:58:59.158 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.158 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.158 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.159 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.159 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:58:59.159 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1865016749999997, 2024-09-13T04:58:59.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T04:58:59.159 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.160 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.160 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.160 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T04:58:59.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.161 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:58:59.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:58:59.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T04:58:59.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T04:58:59.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T04:58:59.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T04:58:59.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T04:58:59.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T04:58:59.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T04:58:59.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T04:58:59.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T04:58:59.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T04:58:59.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T04:58:59.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T04:58:59.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T04:58:59.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T04:58:59.169 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.169 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1826590760000002, 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.170 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:58:59.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T04:58:59.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T04:58:59.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1428110980000001, 2024-09-13T04:58:59.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T04:58:59.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1384818760000002, 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:58:59.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:58:59.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T04:58:59.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1307528759999999, 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.189 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.190 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.190 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:59.191 INFO:teuthology.orchestra.run.smithi001.stdout:3238 read 0x23bdace thru 0x23caa5f (0xcf92 bytes) 2024-09-13T04:58:59.192 INFO:teuthology.orchestra.run.smithi001.stdout:3239 clone 22 order 19 su 262144 sc 9 2024-09-13T04:58:59.193 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:58:59.193 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:59.187+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:59.292 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:58:59.287+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:58:59.366 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:59.366 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:59.366 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:58:59.367 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:59.380 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:58:59.425 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:59.423+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:59.493 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:58:59.491+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:58:59.547 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:59.548 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:58:59.548 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:58:59.548 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:58:59.548 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:59.561 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:58:59.604 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:58:59.603+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:58:59.607 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:58:59.695 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:58:59.691+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:58:59.731 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.694145607000003, 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:58:59.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.694099970000003, 2024-09-13T04:58:59.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:58:59.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:58:59.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:58:59.742 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.742 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.742 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.743 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:59.743 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:59.743 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.694051885999997, 2024-09-13T04:58:59.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:58:59.743 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.743 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.744 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:58:59.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.744 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:59.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:58:59.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:58:59.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:58:59.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:58:59.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:58:59.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:58:59.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:58:59.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:58:59.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:58:59.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:58:59.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:58:59.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:58:59.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:58:59.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:58:59.753 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.753 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.753 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.753 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:59.754 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:59.754 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.693955803999998, 2024-09-13T04:58:59.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:58:59.754 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.754 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.754 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.755 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:58:59.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.755 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:59.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:58:59.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:58:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:58:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:58:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:58:59.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:58:59.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:58:59.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:58:59.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:58:59.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:58:59.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:58:59.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:58:59.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:58:59.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:58:59.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:58:59.763 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.763 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.763 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.764 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:59.764 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:59.764 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.693904433, 2024-09-13T04:58:59.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:58:59.765 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.765 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.765 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:58:59.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.766 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:59.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:58:59.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:58:59.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:58:59.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:58:59.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:58:59.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:58:59.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:58:59.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:58:59.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:58:59.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:58:59.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:58:59.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:58:59.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:58:59.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:58:59.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.774 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.774 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:59.775 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:59.775 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.693858378999998, 2024-09-13T04:58:59.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:58:59.775 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.775 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.776 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:58:59.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.776 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:59.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:58:59.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:58:59.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:58:59.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.782 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:59.782 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:59.782 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.693808318000002, 2024-09-13T04:58:59.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:58:59.782 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.782 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:59.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:58:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:58:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.788 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:58:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.576393127999999, 2024-09-13T04:58:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:58:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:58:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.789 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:58:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:58:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:58:59.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:58:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:58:59.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:58:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:58:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:58:59.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:58:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:58:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:58:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:58:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:58:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:58:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:58:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:58:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:58:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:58:59.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:58:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:58:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:58:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:58:59.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:58:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:58:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:58:59.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:58:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:58:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:58:59.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.802 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484964744999999, 2024-09-13T04:58:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:58:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.803 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:58:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.804 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:58:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484940399999999, 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.809 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:58:59.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:59.813 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484859047, 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:58:59.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:58:59.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484789517000003, 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.818 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484762240999999, 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.822 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.826 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484663796, 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:59.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:58:59.830 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484594229000002, 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.831 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:58:59.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484523754999998, 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.835 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.839 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484498070000001, 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:59.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.843 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484462882000003, 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:58:59.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484438365999999, 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:58:59.848 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.852 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 38.484369029, 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.853 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:58:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:58:59.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:58:59.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:58:59.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:58:59.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:58:59.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.857 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:58:59.857 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:58:59.857 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:58:59.858 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:58:59.858 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:58:59.859 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:58:59.859 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:58:59.860 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:58:59.860 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:58:59.855+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:58:59.860 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:58:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:58:59.861 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:58:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:58:59.861 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:58:59.862 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:58:59.862 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:58:59.862 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:58:59.876 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:58:59.877 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:58:59.897 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:58:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:58:59.959 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:58:59.955+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:58:59.961 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T04:58:59.965 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:58:59.987 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T04:58:59.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:59:00.039 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:00.039 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:00.055 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:00.067 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone21 intact 2024-09-13T04:59:00.104 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:00.103+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:00.152 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-13T04:59:00.161 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:00.159+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:00.162 INFO:teuthology.orchestra.run.smithi001.stdout:3240 trunc from 0x26081f8 to 0x81eb383 2024-09-13T04:59:00.173 INFO:teuthology.orchestra.run.smithi001.stdout:3242 write 0x7802ffe thru 0x7807109 (0x410c bytes) 2024-09-13T04:59:00.175 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:00.176 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:00.192 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:00.192 INFO:teuthology.orchestra.run.smithi001.stdout:3244 trunc from 0x81eb383 to 0x117536 2024-09-13T04:59:00.230 INFO:teuthology.orchestra.run.smithi001.stdout:3245 punch from 0x9ff5d to 0xa59df, (0x5a82 bytes) 2024-09-13T04:59:00.248 INFO:teuthology.orchestra.run.smithi001.stdout:3246 trunc from 0x117536 to 0xcf8bca0 2024-09-13T04:59:00.258 INFO:teuthology.orchestra.run.smithi001.stdout:3247 write 0x10c112f thru 0x10c120c (0xde bytes) 2024-09-13T04:59:00.262 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.262 INFO:teuthology.orchestra.run.smithi001.stdout:3250 writesame 0x902e667 thru 0x903ab91 data_size 0x19(0xc52b bytes) 2024-09-13T04:59:00.263 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:00.259+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:00.269 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.281 INFO:teuthology.orchestra.run.smithi001.stdout:3253 trunc from 0xcf8bca0 to 0x50fb2d4 2024-09-13T04:59:00.294 INFO:teuthology.orchestra.run.smithi001.stdout:3254 read 0x14efdc7 thru 0x14fd710 (0xd94a bytes) 2024-09-13T04:59:00.305 INFO:teuthology.orchestra.run.smithi001.stdout:3255 write 0x758f91f thru 0x759c7c2 (0xcea4 bytes) 2024-09-13T04:59:00.327 INFO:teuthology.orchestra.run.smithi001.stdout:3256 write 0xd127fff thru 0xd132ddd (0xaddf bytes) 2024-09-13T04:59:00.331 INFO:teuthology.orchestra.run.smithi001.stdout:3257 write 0x53ca3dc thru 0x53cb04d (0xc72 bytes) 2024-09-13T04:59:00.335 INFO:teuthology.orchestra.run.smithi001.stdout:3259 write 0x664ae32 thru 0x6656507 (0xb6d6 bytes) 2024-09-13T04:59:00.345 INFO:teuthology.orchestra.run.smithi001.stdout:3260 write 0x448a63c thru 0x448aa72 (0x437 bytes) 2024-09-13T04:59:00.354 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:00.354 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:00.354 INFO:teuthology.orchestra.run.smithi001.stdout:3261 writesame 0xac8e095 thru 0xac93ecf data_size 0x33(0x5e3b bytes) 2024-09-13T04:59:00.356 INFO:teuthology.orchestra.run.smithi001.stdout:3264 write 0x71f1524 thru 0x71f1f02 (0x9df bytes) 2024-09-13T04:59:00.361 INFO:teuthology.orchestra.run.smithi001.stdout:3265 writesame 0xbb18be5 thru 0xbb1dbd7 data_size 0x1(0x4ff3 bytes) 2024-09-13T04:59:00.363 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:00.359+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:00.369 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:00.380 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:59:00.381 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:59:00.381 INFO:teuthology.orchestra.run.smithi001.stdout:3266 read 0x2c67d67 thru 0x2c74296 (0xc530 bytes) 2024-09-13T04:59:00.382 INFO:teuthology.orchestra.run.smithi001.stdout:3268 read 0x4020602 thru 0x4029a69 (0x9468 bytes) 2024-09-13T04:59:00.382 INFO:teuthology.orchestra.run.smithi001.stdout:3269 write 0x1eaf024 thru 0x1ebdb3b (0xeb18 bytes) 2024-09-13T04:59:00.388 INFO:teuthology.orchestra.run.smithi001.stdout:3271 read 0xd065018 thru 0xd06de51 (0x8e3a bytes) 2024-09-13T04:59:00.388 INFO:teuthology.orchestra.run.smithi001.stdout:3272 write 0x8f657eb thru 0x8f6e0c5 (0x88db bytes) 2024-09-13T04:59:00.394 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.410 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-13T04:58:59.958825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666255139,"num_objects":1472,"num_object_clones":340,"num_object_copies":2944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1472,"num_whiteouts":763,"num_read":99691,"num_read_kb":462743,"num_write":10098,"num_write_kb":222737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8250,"ondisk_log_size":8250,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346532,"kb_used_data":44324,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754628188,"statfs":{"total":773094113280,"available":772739264512,"internally_reserved":0,"allocated":45387776,"data_stored":331220620,"data_compressed":9510530,"data_compressed_allocated":30756864,"data_compressed_original":322029620,"omap_allocated":12743,"internal_metadata":309317177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70631123,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":8680,"num_read_kb":55730,"num_write":947,"num_write_kb":18737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.049346"},"pg_stats":[{"pgid":"3.1f","version":"62'296","reported_seq":2255,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.622452+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:58:56.622452+0000","last_peered":"2024-09-13T04:58:56.622452+0000","last_clean":"2024-09-13T04:58:56.622452+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:58:56.622452+0000","last_undegraded":"2024-09-13T04:58:56.622452+0000","last_fullsized":"2024-09-13T04:58:56.622452+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":34399198,"num_objects":58,"num_object_clones":17,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":30,"num_read":3085,"num_read_kb":12464,"num_write":348,"num_write_kb":6794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"58'156","reported_seq":1061,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.255933+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:58:56.255933+0000","last_peered":"2024-09-13T04:58:56.255933+0000","last_clean":"2024-09-13T04:58:56.255933+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:58:56.255933+0000","last_undegraded":"2024-09-13T04:58:56.255933+0000","last_fullsized":"2024-09-13T04:58:56.255933+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":5841927,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":1775,"num_read_kb":7049,"num_write":133,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"63'356","reported_seq":853,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.464555+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:58:57.464555+0000","last_peered":"2024-09-13T04:58:57.464555+0000","last_clean":"2024-09-13T04:58:57.464555+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:58:57.464555+0000","last_undegraded":"2024-09-13T04:58:57.464555+0000","last_fullsized":"2024-09-13T04:58:57.464555+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":30997327,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":1578,"num_read_kb":12057,"num_write":505,"num_write_kb":6386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"63'172","reported_seq":656,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.615274+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:58:57.615274+0000","last_peered":"2024-09-13T04:58:57.615274+0000","last_clean":"2024-09-13T04:58:57.615274+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:58:57.615274+0000","last_undegraded":"2024-09-13T04:58:57.615274+0000","last_fullsized":"2024-09-13T04:58:57.615274+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":8358552,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":542,"num_read_kb":5316,"num_write":152,"num_write_kb":5419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"63'289","reported_seq":4413,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.409153+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:58:57.409153+0000","last_peered":"2024-09-13T04:58:57.409153+0000","last_clean":"2024-09-13T04:58:57.409153+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:58:57.409153+0000","last_undegraded":"2024-09-13T04:58:57.409153+0000","last_fullsized":"2024-09-13T04:58:57.409153+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":19284046,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":5166,"num_read_kb":12176,"num_write":340,"num_write_kb":5200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"63'133","reported_seq":1566,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.848962+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:58:57.848962+0000","last_peered":"2024-09-13T04:58:57.848962+0000","last_clean":"2024-09-13T04:58:57.848962+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:58:57.848962+0000","last_undegraded":"2024-09-13T04:58:57.848962+0000","last_fullsized":"2024-09-13T04:58:57.848962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":18448845,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":1520,"num_read_kb":7395,"num_write":130,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"63'302","reported_seq":17492,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.824934+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:58:57.824934+0000","last_peered":"2024-09-13T04:58:57.824934+0000","last_clean":"2024-09-13T04:58:57.824934+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:58:57.824934+0000","last_undegraded":"2024-09-13T04:58:57.824934+0000","last_fullsized":"2024-09-13T04:58:57.824934+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":45843261,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":30,"num_read":18186,"num_read_kb":92793,"num_write":365,"num_write_kb":18063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"62'154","reported_seq":2180,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.910875+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:58:56.910875+0000","last_peered":"2024-09-13T04:58:56.910875+0000","last_clean":"2024-09-13T04:58:56.910875+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:58:56.910875+0000","last_undegraded":"2024-09-13T04:58:56.910875+0000","last_fullsized":"2024-09-13T04:58:56.910875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":24045220,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":26,"num_read":2076,"num_read_kb":5637,"num_write":137,"num_write_kb":2467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"57'150","reported_seq":2800,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.246052+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:58:56.246052+0000","last_peered":"2024-09-13T04:58:56.246052+0000","last_clean":"2024-09-13T04:58:56.246052+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:58:56.246052+0000","last_undegraded":"2024-09-13T04:58:56.246052+0000","last_fullsized":"2024-09-13T04:58:56.246052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":16690695,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":25,"num_read":2654,"num_read_kb":20419,"num_write":164,"num_write_kb":12572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"63'325","reported_seq":742,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.615705+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:58:57.615705+0000","last_peered":"2024-09-13T04:58:57.615705+0000","last_clean":"2024-09-13T04:58:57.615705+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:58:57.615705+0000","last_undegraded":"2024-09-13T04:58:57.615705+0000","last_fullsized":"2024-09-13T04:58:57.615705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":7207272,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":1427,"num_read_kb":7008,"num_write":447,"num_write_kb":2698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"63'199","reported_seq":436,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.214821+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:58:57.214821+0000","last_peered":"2024-09-13T04:58:57.214821+0000","last_clean":"2024-09-13T04:58:57.214821+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:58:57.214821+0000","last_undegraded":"2024-09-13T04:58:57.214821+0000","last_fullsized":"2024-09-13T04:58:57.214821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":14084194,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":554,"num_read_kb":8561,"num_write":238,"num_write_kb":4639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"63'332","reported_seq":2914,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.821728+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:58:57.821728+0000","last_peered":"2024-09-13T04:58:57.821728+0000","last_clean":"2024-09-13T04:58:57.821728+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:58:57.821728+0000","last_undegraded":"2024-09-13T04:58:57.821728+0000","last_fullsized":"2024-09-13T04:58:57.821728+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":18442237,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":3635,"num_read_kb":11539,"num_write":432,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939873+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:58:56.939873+0000","last_peered":"2024-09-13T04:58:56.939873+0000","last_clean":"2024-09-13T04:58:56.939873+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:58:56.939873+0000","last_undegraded":"2024-09-13T04:58:56.939873+0000","last_fullsized":"2024-09-13T04:58:56.939873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'155","reported_seq":2133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.797142+0000","last_change":"2024-09-13T04:58:19.436873+0000","last_active":"2024-09-13T04:58:57.797142+0000","last_peered":"2024-09-13T04:58:57.797142+0000","last_clean":"2024-09-13T04:58:57.797142+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T04:58:57.797142+0000","last_undegraded":"2024-09-13T04:58:57.797142+0000","last_fullsized":"2024-09-13T04:58:57.797142+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":42641086,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":2016,"num_read_kb":18107,"num_write":166,"num_write_kb":17284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:55.926320+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:58:55.926320+0000","last_peered":"2024-09-13T04:58:55.926320+0000","last_clean":"2024-09-13T04:58:55.926320+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:58:55.926320+0000","last_undegraded":"2024-09-13T04:58:55.926320+0000","last_fullsized":"2024-09-13T04:58:55.926320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'258","reported_seq":1211,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.665522+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:58:57.665522+0000","last_peered":"2024-09-13T04:58:57.665522+0000","last_clean":"2024-09-13T04:58:57.665522+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:58:57.665522+0000","last_undegraded":"2024-09-13T04:58:57.665522+0000","last_fullsized":"2024-09-13T04:58:57.665522+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":16683193,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":2232,"num_read_kb":9815,"num_write":286,"num_write_kb":4686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939820+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:58:56.939820+0000","last_peered":"2024-09-13T04:58:56.939820+0000","last_clean":"2024-09-13T04:58:56.939820+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:58:56.939820+0000","last_undegraded":"2024-09-13T04:58:56.939820+0000","last_fullsized":"2024-09-13T04:58:56.939820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"63'380","reported_seq":566,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.835042+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:58:57.835042+0000","last_peered":"2024-09-13T04:58:57.835042+0000","last_clean":"2024-09-13T04:58:57.835042+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:58:57.835042+0000","last_undegraded":"2024-09-13T04:58:57.835042+0000","last_fullsized":"2024-09-13T04:58:57.835042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":14898251,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":28,"num_read":1443,"num_read_kb":6140,"num_write":564,"num_write_kb":5019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:55.960537+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:58:55.960537+0000","last_peered":"2024-09-13T04:58:55.960537+0000","last_clean":"2024-09-13T04:58:55.960537+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:58:55.960537+0000","last_undegraded":"2024-09-13T04:58:55.960537+0000","last_fullsized":"2024-09-13T04:58:55.960537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"63'311","reported_seq":909,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.762531+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:58:57.762531+0000","last_peered":"2024-09-13T04:58:57.762531+0000","last_clean":"2024-09-13T04:58:57.762531+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:58:57.762531+0000","last_undegraded":"2024-09-13T04:58:57.762531+0000","last_fullsized":"2024-09-13T04:58:57.762531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":10263090,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":25,"num_read":1857,"num_read_kb":7437,"num_write":382,"num_write_kb":3623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"63'262","reported_seq":2578,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.805313+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:58:57.805313+0000","last_peered":"2024-09-13T04:58:57.805313+0000","last_clean":"2024-09-13T04:58:57.805313+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:58:57.805313+0000","last_undegraded":"2024-09-13T04:58:57.805313+0000","last_fullsized":"2024-09-13T04:58:57.805313+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":38869652,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":29,"num_read":3107,"num_read_kb":16272,"num_write":292,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939559+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:58:56.939559+0000","last_peered":"2024-09-13T04:58:56.939559+0000","last_clean":"2024-09-13T04:58:56.939559+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:58:56.939559+0000","last_undegraded":"2024-09-13T04:58:56.939559+0000","last_fullsized":"2024-09-13T04:58:56.939559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"62'377","reported_seq":579,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.634925+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:58:56.634925+0000","last_peered":"2024-09-13T04:58:56.634925+0000","last_clean":"2024-09-13T04:58:56.634925+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:58:56.634925+0000","last_undegraded":"2024-09-13T04:58:56.634925+0000","last_fullsized":"2024-09-13T04:58:56.634925+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":10807072,"num_objects":42,"num_object_clones":5,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":1336,"num_read_kb":3918,"num_write":520,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.972229+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:58:56.972229+0000","last_peered":"2024-09-13T04:58:56.972229+0000","last_clean":"2024-09-13T04:58:56.972229+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:58:56.972229+0000","last_undegraded":"2024-09-13T04:58:56.972229+0000","last_fullsized":"2024-09-13T04:58:56.972229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"63'164","reported_seq":3981,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.647402+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:58:57.647402+0000","last_peered":"2024-09-13T04:58:57.647402+0000","last_clean":"2024-09-13T04:58:57.647402+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:58:57.647402+0000","last_undegraded":"2024-09-13T04:58:57.647402+0000","last_fullsized":"2024-09-13T04:58:57.647402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":39021436,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":4162,"num_read_kb":20849,"num_write":156,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939347+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:58:56.939347+0000","last_peered":"2024-09-13T04:58:56.939347+0000","last_clean":"2024-09-13T04:58:56.939347+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:58:56.939347+0000","last_undegraded":"2024-09-13T04:58:56.939347+0000","last_fullsized":"2024-09-13T04:58:56.939347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"63'215","reported_seq":1295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.114774+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:58:57.114774+0000","last_peered":"2024-09-13T04:58:57.114774+0000","last_clean":"2024-09-13T04:58:57.114774+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:58:57.114774+0000","last_undegraded":"2024-09-13T04:58:57.114774+0000","last_fullsized":"2024-09-13T04:58:57.114774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":16589690,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":2112,"num_read_kb":16086,"num_write":195,"num_write_kb":6276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":58,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939646+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:58:56.939646+0000","last_peered":"2024-09-13T04:58:56.939646+0000","last_clean":"2024-09-13T04:58:56.939646+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:58:56.939646+0000","last_undegraded":"2024-09-13T04:58:56.939646+0000","last_fullsized":"2024-09-13T04:58:56.939646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.942761+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:58:56.942761+0000","last_peered":"2024-09-13T04:58:56.942761+0000","last_clean":"2024-09-13T04:58:56.942761+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:58:56.942761+0000","last_undegraded":"2024-09-13T04:58:56.942761+0000","last_fullsized":"2024-09-13T04:58:56.942761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"63'132","reported_seq":724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.430002+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:58:57.430002+0000","last_peered":"2024-09-13T04:58:57.430002+0000","last_clean":"2024-09-13T04:58:57.430002+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:58:57.430002+0000","last_undegraded":"2024-09-13T04:58:57.430002+0000","last_fullsized":"2024-09-13T04:58:57.430002+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":5659465,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":1075,"num_read_kb":2545,"num_write":111,"num_write_kb":1540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"63'388","reported_seq":5068,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.831063+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:58:57.831063+0000","last_peered":"2024-09-13T04:58:57.831063+0000","last_clean":"2024-09-13T04:58:57.831063+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:58:57.831063+0000","last_undegraded":"2024-09-13T04:58:57.831063+0000","last_fullsized":"2024-09-13T04:58:57.831063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":19561835,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":6175,"num_read_kb":20265,"num_write":481,"num_write_kb":9342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"63'191","reported_seq":4531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.805660+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:58:57.805660+0000","last_peered":"2024-09-13T04:58:57.805660+0000","last_clean":"2024-09-13T04:58:57.805660+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:58:57.805660+0000","last_undegraded":"2024-09-13T04:58:57.805660+0000","last_fullsized":"2024-09-13T04:58:57.805660+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":16481506,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":4529,"num_read_kb":17046,"num_write":206,"num_write_kb":11156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"63'153","reported_seq":549,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.736521+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:58:57.736521+0000","last_peered":"2024-09-13T04:58:57.736521+0000","last_clean":"2024-09-13T04:58:57.736521+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:58:57.736521+0000","last_undegraded":"2024-09-13T04:58:57.736521+0000","last_fullsized":"2024-09-13T04:58:57.736521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":28294364,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":411,"num_read_kb":11466,"num_write":146,"num_write_kb":8239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"63'408","reported_seq":829,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.512564+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:58:57.512564+0000","last_peered":"2024-09-13T04:58:57.512564+0000","last_clean":"2024-09-13T04:58:57.512564+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:58:57.512564+0000","last_undegraded":"2024-09-13T04:58:57.512564+0000","last_fullsized":"2024-09-13T04:58:57.512564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":13567710,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":1839,"num_read_kb":6279,"num_write":622,"num_write_kb":4038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"63'225","reported_seq":3403,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.562423+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:58:57.562423+0000","last_peered":"2024-09-13T04:58:57.562423+0000","last_clean":"2024-09-13T04:58:57.562423+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:58:57.562423+0000","last_undegraded":"2024-09-13T04:58:57.562423+0000","last_fullsized":"2024-09-13T04:58:57.562423+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":28011766,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":4390,"num_read_kb":28951,"num_write":385,"num_write_kb":9503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"63'158","reported_seq":602,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.822983+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:58:57.822983+0000","last_peered":"2024-09-13T04:58:57.822983+0000","last_clean":"2024-09-13T04:58:57.822983+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:58:57.822983+0000","last_undegraded":"2024-09-13T04:58:57.822983+0000","last_fullsized":"2024-09-13T04:58:57.822983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":18062207,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":21,"num_read":753,"num_read_kb":6498,"num_write":141,"num_write_kb":2881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"63'438","reported_seq":5389,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.758533+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:58:57.758533+0000","last_peered":"2024-09-13T04:58:57.758533+0000","last_clean":"2024-09-13T04:58:57.758533+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:58:57.758533+0000","last_undegraded":"2024-09-13T04:58:57.758533+0000","last_fullsized":"2024-09-13T04:58:57.758533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":20263066,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":30,"num_read":6458,"num_read_kb":20158,"num_write":653,"num_write_kb":11669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"63'226","reported_seq":4274,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.997077+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:58:56.997077+0000","last_peered":"2024-09-13T04:58:56.997077+0000","last_clean":"2024-09-13T04:58:56.997077+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:58:56.997077+0000","last_undegraded":"2024-09-13T04:58:56.997077+0000","last_fullsized":"2024-09-13T04:58:56.997077+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":14841877,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":36,"num_read":4773,"num_read_kb":6258,"num_write":214,"num_write_kb":891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"63'219","reported_seq":3895,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.849837+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:58:57.849837+0000","last_peered":"2024-09-13T04:58:57.849837+0000","last_clean":"2024-09-13T04:58:57.849837+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:58:57.849837+0000","last_undegraded":"2024-09-13T04:58:57.849837+0000","last_fullsized":"2024-09-13T04:58:57.849837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":43920794,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":23,"num_read":4139,"num_read_kb":26995,"num_write":229,"num_write_kb":15429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'178","reported_seq":1905,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.822484+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:58:57.822484+0000","last_peered":"2024-09-13T04:58:57.822484+0000","last_clean":"2024-09-13T04:58:57.822484+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:58:57.822484+0000","last_undegraded":"2024-09-13T04:58:57.822484+0000","last_fullsized":"2024-09-13T04:58:57.822484+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":12813357,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":1835,"num_read_kb":6905,"num_write":194,"num_write_kb":5454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"63'428","reported_seq":1181,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.313035+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:58:57.313035+0000","last_peered":"2024-09-13T04:58:57.313035+0000","last_clean":"2024-09-13T04:58:57.313035+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:58:57.313035+0000","last_undegraded":"2024-09-13T04:58:57.313035+0000","last_fullsized":"2024-09-13T04:58:57.313035+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":10770561,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":2735,"num_read_kb":8239,"num_write":603,"num_write_kb":4197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":665664752,"num_objects":1468,"num_object_clones":340,"num_object_copies":2936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1468,"num_whiteouts":763,"num_read":99575,"num_read_kb":462643,"num_write":9937,"num_write_kb":220319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39641088,"data_stored":327859653,"data_compressed":9555545,"data_compressed_allocated":30781440,"data_compressed_original":321218612,"omap_allocated":0,"internal_metadata":0},"log_size":8130,"ondisk_log_size":8130,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2752512,"data_stored":15301926,"data_compressed":514944,"data_compressed_allocated":1458176,"data_compressed_original":14572157,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.75900000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.876}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.044},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.0720000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.1890000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.117}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.258}]}]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40892,"kb_used_data":5680,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330948,"statfs":{"total":96636764160,"available":96594890752,"internally_reserved":0,"allocated":5816320,"data_stored":43980367,"data_compressed":1096454,"data_compressed_allocated":3846144,"data_compressed_original":42739025,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.022}]},{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.878},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.224}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.1879999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41084,"kb_used_data":5868,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330756,"statfs":{"total":96636764160,"available":96594694144,"internally_reserved":0,"allocated":6008832,"data_stored":45424984,"data_compressed":1469043,"data_compressed_allocated":4407296,"data_compressed_original":44463476,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.79200000000000004}]},{"osd":3,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":11.266,"5min":11.266,"15min":11.266},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.0169999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.76000000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50508,"kb_used_data":7100,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94321332,"statfs":{"total":96636764160,"available":96585043968,"internally_reserved":0,"allocated":7270400,"data_stored":54419661,"data_compressed":1376219,"data_compressed_allocated":5148672,"data_compressed_original":53018427,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.47},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.552}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.583}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.6180000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.7170000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":2.1499999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.9550000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":2.2229999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.879}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.796},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":2.0099999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44452,"kb_used_data":5144,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327388,"statfs":{"total":96636764160,"available":96591245312,"internally_reserved":0,"allocated":5267456,"data_stored":41872836,"data_compressed":1226689,"data_compressed_allocated":3555328,"data_compressed_original":40823847,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.378}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.23400000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.48299999999999998}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.503},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.54800000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46956,"kb_used_data":7648,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94324884,"statfs":{"total":96636764160,"available":96588681216,"internally_reserved":0,"allocated":7831552,"data_stored":57081868,"data_compressed":1503531,"data_compressed_allocated":5767168,"data_compressed_original":55773714,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.008},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.103}]},{"osd":6,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.92400000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.96499999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49028,"kb_used_data":5620,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94322812,"statfs":{"total":96636764160,"available":96586559488,"internally_reserved":0,"allocated":5754880,"data_stored":42934214,"data_compressed":1288933,"data_compressed_allocated":3780608,"data_compressed_original":41640796,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":15.755000000000001,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":15.802,"5min":15.802,"15min":15.802},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1,"5min":1,"15min":1},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":1.794,"15min":1.794},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":15.859999999999999,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":11.474,"5min":11.474,"15min":11.474},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":11.528,"5min":11.528,"15min":11.528},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.897,"5min":1.897,"15min":1.897},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":15.885999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":1.012}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":15.834,"5min":15.834,"15min":15.834},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.95899999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39796,"kb_used_data":4576,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332044,"statfs":{"total":96636764160,"available":96596013056,"internally_reserved":0,"allocated":4685824,"data_stored":30204764,"data_compressed":1034717,"data_compressed_allocated":2793472,"data_compressed_original":28998178,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.84699999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":1.1519999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.633,"5min":4.633,"15min":4.633},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.6989999999999998,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.874}]},{"osd":7,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.46200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5046272,"data_stored":42601699,"data_compressed":1294200,"data_compressed_allocated":3780608,"data_compressed_original":41620316,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7024640,"data_stored":56120020,"data_compressed":1481249,"data_compressed_allocated":5709824,"data_compressed_original":55151122,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4567040,"data_stored":41617305,"data_compressed":1226987,"data_compressed_allocated":3563520,"data_compressed_original":40881191,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6586368,"data_stored":54184160,"data_compressed":1393018,"data_compressed_allocated":5181440,"data_compressed_original":53104443,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5308416,"data_stored":45140146,"data_compressed":1485269,"data_compressed_allocated":4423680,"data_compressed_original":44488052,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3932160,"data_stored":29431857,"data_compressed":1040978,"data_compressed_allocated":2772992,"data_compressed_original":28555810,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5169152,"data_stored":43782697,"data_compressed":1118706,"data_compressed_allocated":3887104,"data_compressed_original":42833233,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2007040,"data_stored":14981769,"data_compressed":515138,"data_compressed_allocated":1462272,"data_compressed_original":14584445,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:59:00.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:59:00.414 INFO:teuthology.orchestra.run.smithi001.stdout:3274 trunc from 0xd132dde to 0x4269994 2024-09-13T04:59:00.429 INFO:teuthology.orchestra.run.smithi001.stdout:3275 write 0x5247633 thru 0x524b1c7 (0x3b95 bytes) 2024-09-13T04:59:00.432 INFO:teuthology.orchestra.run.smithi001.stdout:3276 punch from 0x1a99ec1 to 0x1aa2703, (0x8842 bytes) 2024-09-13T04:59:00.446 INFO:teuthology.orchestra.run.smithi001.stdout:3278 write 0x94b6e48 thru 0x94b79e5 (0xb9e bytes) 2024-09-13T04:59:00.450 INFO:teuthology.orchestra.run.smithi001.stdout:3279 read 0x3cb2d16 thru 0x3cbeedb (0xc1c6 bytes) 2024-09-13T04:59:00.450 INFO:teuthology.orchestra.run.smithi001.stdout:3280 punch from 0x312b6ef to 0x312ebc7, (0x34d8 bytes) 2024-09-13T04:59:00.450 INFO:teuthology.orchestra.run.smithi001.stdout:3281 writesame 0x4354995 thru 0x43648eb data_size 0x1b(0xff57 bytes) 2024-09-13T04:59:00.457 INFO:teuthology.orchestra.run.smithi001.stdout:3282 trunc from 0x94b79e6 to 0xa443ac6 2024-09-13T04:59:00.464 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:00.463+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:00.473 INFO:teuthology.orchestra.run.smithi001.stdout:3284 write 0xb7d22c5 thru 0xb7dfcf7 (0xda33 bytes) 2024-09-13T04:59:00.479 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.479 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.479 INFO:teuthology.orchestra.run.smithi001.stdout:3288 write 0x31fa568 thru 0x3200b90 (0x6629 bytes) 2024-09-13T04:59:00.491 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:00.491 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:00.495 INFO:teuthology.orchestra.run.smithi001.stdout:3289 trunc from 0xb7dfcf8 to 0x626f009 2024-09-13T04:59:00.505 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:00.509 INFO:teuthology.orchestra.run.smithi001.stdout:3290 punch from 0x47ff3a7 to 0x48079c8, (0x8621 bytes) 2024-09-13T04:59:00.509 INFO:teuthology.orchestra.run.smithi001.stdout:3291 writesame 0x13a10c4 thru 0x13b089b data_size 0x67(0xf7d8 bytes) 2024-09-13T04:59:00.513 INFO:teuthology.orchestra.run.smithi001.stdout:3292 punch from 0x51533c1 to 0x515d417, (0xa056 bytes) 2024-09-13T04:59:00.513 INFO:teuthology.orchestra.run.smithi001.stdout:3293 write 0x346108b thru 0x346f596 (0xe50c bytes) 2024-09-13T04:59:00.522 INFO:teuthology.orchestra.run.smithi001.stdout:3294 punch from 0x6eb56c to 0x6eddc5, (0x2859 bytes) 2024-09-13T04:59:00.535 INFO:teuthology.orchestra.run.smithi001.stdout:3295 write 0xa3dce5a thru 0xa3ecd27 (0xfece bytes) 2024-09-13T04:59:00.539 INFO:teuthology.orchestra.run.smithi001.stdout:3296 punch from 0x49ae3da to 0x49b810b, (0x9d31 bytes) 2024-09-13T04:59:00.552 INFO:teuthology.orchestra.run.smithi001.stdout:3297 write 0xe785e39 thru 0xe786ba6 (0xd6e bytes) 2024-09-13T04:59:00.562 INFO:teuthology.orchestra.run.smithi001.stdout:3298 write 0xeb28ca1 thru 0xeb2f912 (0x6c72 bytes) 2024-09-13T04:59:00.583 INFO:teuthology.orchestra.run.smithi001.stdout:3300 trunc from 0xeb2f913 to 0x2de72b7 2024-09-13T04:59:00.592 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:00.591+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:00.620 INFO:teuthology.orchestra.run.smithi001.stdout:3301 writesame 0xee3f6d0 thru 0xee46d98 data_size 0x1(0x76c9 bytes) 2024-09-13T04:59:00.620 INFO:teuthology.orchestra.run.smithi001.stdout:3304 read 0x4d7228c thru 0x4d820c1 (0xfe36 bytes) 2024-09-13T04:59:00.620 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.620 INFO:teuthology.orchestra.run.smithi001.stdout:3306 writesame 0x4a97337 thru 0x4a9be27 data_size 0x5(0x4af1 bytes) 2024-09-13T04:59:00.626 INFO:teuthology.orchestra.run.smithi001.stdout:3307 read 0xb113976 thru 0xb1182fe (0x4989 bytes) 2024-09-13T04:59:00.626 INFO:teuthology.orchestra.run.smithi001.stdout:3308 write 0x24b37a4 thru 0x24be639 (0xae96 bytes) 2024-09-13T04:59:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:3309 punch from 0xeb5aef2 to 0xeb659cc, (0xaada bytes) 2024-09-13T04:59:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:3315 punch from 0x45db81a to 0x45dcf50, (0x1736 bytes) 2024-09-13T04:59:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:3317 punch from 0x8e3a6d7 to 0x8e48d55, (0xe67e bytes) 2024-09-13T04:59:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:3318 read 0x71001e thru 0x712f18 (0x2efb bytes) 2024-09-13T04:59:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:3319 write 0xaecf1bb thru 0xaed7816 (0x865c bytes) 2024-09-13T04:59:00.641 INFO:teuthology.orchestra.run.smithi001.stdout:3320 write 0x555af2e thru 0x5563333 (0x8406 bytes) 2024-09-13T04:59:00.650 INFO:teuthology.orchestra.run.smithi001.stdout:3321 read 0x5e3c814 thru 0x5e3d9b7 (0x11a4 bytes) 2024-09-13T04:59:00.663 INFO:teuthology.orchestra.run.smithi001.stdout:3322 trunc from 0xee46d99 to 0x69efb28 2024-09-13T04:59:00.665 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:00.663+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:00.674 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:00.674 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:00.678 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.684 INFO:teuthology.orchestra.run.smithi001.stdout:3324 trunc from 0x69efb28 to 0x27c6cdc 2024-09-13T04:59:00.688 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:00.700 INFO:teuthology.orchestra.run.smithi001.stdout:3325 trunc from 0x27c6cdc to 0x13c8172 2024-09-13T04:59:00.741 INFO:teuthology.orchestra.run.smithi001.stdout:3326 write 0xab1ed58 thru 0xab28da0 (0xa049 bytes) 2024-09-13T04:59:00.747 INFO:teuthology.orchestra.run.smithi001.stdout:3327 writesame 0x8646e0a thru 0x86525a6 data_size 0x55(0xb79d bytes) 2024-09-13T04:59:00.753 INFO:teuthology.orchestra.run.smithi001.stdout:3328 read 0x5dade9b thru 0x5dba6dc (0xc842 bytes) 2024-09-13T04:59:00.754 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.760 INFO:teuthology.orchestra.run.smithi001.stdout:3330 trunc from 0xab28da1 to 0x65efa30 2024-09-13T04:59:00.778 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:59:00.780 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:59:00.780 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:00.779+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:00.791 INFO:teuthology.orchestra.run.smithi001.stdout:3332 write 0xa56de89 thru 0xa570357 (0x24cf bytes) 2024-09-13T04:59:00.796 INFO:teuthology.orchestra.run.smithi001.stdout:3333 punch from 0x9c263d5 to 0x9c2a38d, (0x3fb8 bytes) 2024-09-13T04:59:00.796 INFO:teuthology.orchestra.run.smithi001.stdout:3334 read 0x420260 thru 0x421849 (0x15ea bytes) 2024-09-13T04:59:00.796 INFO:teuthology.orchestra.run.smithi001.stdout:3337 write 0xd9281f thru 0xd959a5 (0x3187 bytes) 2024-09-13T04:59:00.801 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-13T04:58:59.958825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666255139,"num_objects":1472,"num_object_clones":340,"num_object_copies":2944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1472,"num_whiteouts":763,"num_read":99691,"num_read_kb":462743,"num_write":10098,"num_write_kb":222737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8250,"ondisk_log_size":8250,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346532,"kb_used_data":44324,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754628188,"statfs":{"total":773094113280,"available":772739264512,"internally_reserved":0,"allocated":45387776,"data_stored":331220620,"data_compressed":9510530,"data_compressed_allocated":30756864,"data_compressed_original":322029620,"omap_allocated":12743,"internal_metadata":309317177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70631123,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":8680,"num_read_kb":55730,"num_write":947,"num_write_kb":18737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.049346"},"pg_stats":[{"pgid":"3.1f","version":"62'296","reported_seq":2255,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.622452+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:58:56.622452+0000","last_peered":"2024-09-13T04:58:56.622452+0000","last_clean":"2024-09-13T04:58:56.622452+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:58:56.622452+0000","last_undegraded":"2024-09-13T04:58:56.622452+0000","last_fullsized":"2024-09-13T04:58:56.622452+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":34399198,"num_objects":58,"num_object_clones":17,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":30,"num_read":3085,"num_read_kb":12464,"num_write":348,"num_write_kb":6794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"58'156","reported_seq":1061,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.255933+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:58:56.255933+0000","last_peered":"2024-09-13T04:58:56.255933+0000","last_clean":"2024-09-13T04:58:56.255933+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:58:56.255933+0000","last_undegraded":"2024-09-13T04:58:56.255933+0000","last_fullsized":"2024-09-13T04:58:56.255933+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":5841927,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":1775,"num_read_kb":7049,"num_write":133,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"63'356","reported_seq":853,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.464555+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:58:57.464555+0000","last_peered":"2024-09-13T04:58:57.464555+0000","last_clean":"2024-09-13T04:58:57.464555+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:58:57.464555+0000","last_undegraded":"2024-09-13T04:58:57.464555+0000","last_fullsized":"2024-09-13T04:58:57.464555+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":30997327,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":1578,"num_read_kb":12057,"num_write":505,"num_write_kb":6386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"63'172","reported_seq":656,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.615274+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:58:57.615274+0000","last_peered":"2024-09-13T04:58:57.615274+0000","last_clean":"2024-09-13T04:58:57.615274+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:58:57.615274+0000","last_undegraded":"2024-09-13T04:58:57.615274+0000","last_fullsized":"2024-09-13T04:58:57.615274+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":8358552,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":542,"num_read_kb":5316,"num_write":152,"num_write_kb":5419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"63'289","reported_seq":4413,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.409153+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:58:57.409153+0000","last_peered":"2024-09-13T04:58:57.409153+0000","last_clean":"2024-09-13T04:58:57.409153+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:58:57.409153+0000","last_undegraded":"2024-09-13T04:58:57.409153+0000","last_fullsized":"2024-09-13T04:58:57.409153+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":19284046,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":5166,"num_read_kb":12176,"num_write":340,"num_write_kb":5200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"63'133","reported_seq":1566,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.848962+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:58:57.848962+0000","last_peered":"2024-09-13T04:58:57.848962+0000","last_clean":"2024-09-13T04:58:57.848962+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:58:57.848962+0000","last_undegraded":"2024-09-13T04:58:57.848962+0000","last_fullsized":"2024-09-13T04:58:57.848962+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":18448845,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":1520,"num_read_kb":7395,"num_write":130,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"63'302","reported_seq":17492,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.824934+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:58:57.824934+0000","last_peered":"2024-09-13T04:58:57.824934+0000","last_clean":"2024-09-13T04:58:57.824934+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:58:57.824934+0000","last_undegraded":"2024-09-13T04:58:57.824934+0000","last_fullsized":"2024-09-13T04:58:57.824934+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":45843261,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":30,"num_read":18186,"num_read_kb":92793,"num_write":365,"num_write_kb":18063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"62'154","reported_seq":2180,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.910875+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:58:56.910875+0000","last_peered":"2024-09-13T04:58:56.910875+0000","last_clean":"2024-09-13T04:58:56.910875+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:58:56.910875+0000","last_undegraded":"2024-09-13T04:58:56.910875+0000","last_fullsized":"2024-09-13T04:58:56.910875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":24045220,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":26,"num_read":2076,"num_read_kb":5637,"num_write":137,"num_write_kb":2467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"57'150","reported_seq":2800,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.246052+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:58:56.246052+0000","last_peered":"2024-09-13T04:58:56.246052+0000","last_clean":"2024-09-13T04:58:56.246052+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:58:56.246052+0000","last_undegraded":"2024-09-13T04:58:56.246052+0000","last_fullsized":"2024-09-13T04:58:56.246052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":16690695,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":25,"num_read":2654,"num_read_kb":20419,"num_write":164,"num_write_kb":12572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"63'325","reported_seq":742,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.615705+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:58:57.615705+0000","last_peered":"2024-09-13T04:58:57.615705+0000","last_clean":"2024-09-13T04:58:57.615705+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:58:57.615705+0000","last_undegraded":"2024-09-13T04:58:57.615705+0000","last_fullsized":"2024-09-13T04:58:57.615705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":7207272,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":1427,"num_read_kb":7008,"num_write":447,"num_write_kb":2698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"63'199","reported_seq":436,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.214821+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:58:57.214821+0000","last_peered":"2024-09-13T04:58:57.214821+0000","last_clean":"2024-09-13T04:58:57.214821+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:58:57.214821+0000","last_undegraded":"2024-09-13T04:58:57.214821+0000","last_fullsized":"2024-09-13T04:58:57.214821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":14084194,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":554,"num_read_kb":8561,"num_write":238,"num_write_kb":4639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"63'332","reported_seq":2914,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.821728+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:58:57.821728+0000","last_peered":"2024-09-13T04:58:57.821728+0000","last_clean":"2024-09-13T04:58:57.821728+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:58:57.821728+0000","last_undegraded":"2024-09-13T04:58:57.821728+0000","last_fullsized":"2024-09-13T04:58:57.821728+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":18442237,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":3635,"num_read_kb":11539,"num_write":432,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939873+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:58:56.939873+0000","last_peered":"2024-09-13T04:58:56.939873+0000","last_clean":"2024-09-13T04:58:56.939873+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:58:56.939873+0000","last_undegraded":"2024-09-13T04:58:56.939873+0000","last_fullsized":"2024-09-13T04:58:56.939873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'155","reported_seq":2133,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.797142+0000","last_change":"2024-09-13T04:58:19.436873+0000","last_active":"2024-09-13T04:58:57.797142+0000","last_peered":"2024-09-13T04:58:57.797142+0000","last_clean":"2024-09-13T04:58:57.797142+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T04:58:57.797142+0000","last_undegraded":"2024-09-13T04:58:57.797142+0000","last_fullsized":"2024-09-13T04:58:57.797142+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":42641086,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":2016,"num_read_kb":18107,"num_write":166,"num_write_kb":17284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:55.926320+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:58:55.926320+0000","last_peered":"2024-09-13T04:58:55.926320+0000","last_clean":"2024-09-13T04:58:55.926320+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:58:55.926320+0000","last_undegraded":"2024-09-13T04:58:55.926320+0000","last_fullsized":"2024-09-13T04:58:55.926320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'258","reported_seq":1211,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.665522+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:58:57.665522+0000","last_peered":"2024-09-13T04:58:57.665522+0000","last_clean":"2024-09-13T04:58:57.665522+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:58:57.665522+0000","last_undegraded":"2024-09-13T04:58:57.665522+0000","last_fullsized":"2024-09-13T04:58:57.665522+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":16683193,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":2232,"num_read_kb":9815,"num_write":286,"num_write_kb":4686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939820+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:58:56.939820+0000","last_peered":"2024-09-13T04:58:56.939820+0000","last_clean":"2024-09-13T04:58:56.939820+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:58:56.939820+0000","last_undegraded":"2024-09-13T04:58:56.939820+0000","last_fullsized":"2024-09-13T04:58:56.939820+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"63'380","reported_seq":566,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.835042+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:58:57.835042+0000","last_peered":"2024-09-13T04:58:57.835042+0000","last_clean":"2024-09-13T04:58:57.835042+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:58:57.835042+0000","last_undegraded":"2024-09-13T04:58:57.835042+0000","last_fullsized":"2024-09-13T04:58:57.835042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":14898251,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":28,"num_read":1443,"num_read_kb":6140,"num_write":564,"num_write_kb":5019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":56,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:55.960537+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:58:55.960537+0000","last_peered":"2024-09-13T04:58:55.960537+0000","last_clean":"2024-09-13T04:58:55.960537+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:58:55.960537+0000","last_undegraded":"2024-09-13T04:58:55.960537+0000","last_fullsized":"2024-09-13T04:58:55.960537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"63'311","reported_seq":909,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.762531+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:58:57.762531+0000","last_peered":"2024-09-13T04:58:57.762531+0000","last_clean":"2024-09-13T04:58:57.762531+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:58:57.762531+0000","last_undegraded":"2024-09-13T04:58:57.762531+0000","last_fullsized":"2024-09-13T04:58:57.762531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":10263090,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":25,"num_read":1857,"num_read_kb":7437,"num_write":382,"num_write_kb":3623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"63'262","reported_seq":2578,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.805313+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:58:57.805313+0000","last_peered":"2024-09-13T04:58:57.805313+0000","last_clean":"2024-09-13T04:58:57.805313+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:58:57.805313+0000","last_undegraded":"2024-09-13T04:58:57.805313+0000","last_fullsized":"2024-09-13T04:58:57.805313+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":38869652,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":29,"num_read":3107,"num_read_kb":16272,"num_write":292,"num_write_kb":13034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939559+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:58:56.939559+0000","last_peered":"2024-09-13T04:58:56.939559+0000","last_clean":"2024-09-13T04:58:56.939559+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:58:56.939559+0000","last_undegraded":"2024-09-13T04:58:56.939559+0000","last_fullsized":"2024-09-13T04:58:56.939559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"62'377","reported_seq":579,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.634925+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:58:56.634925+0000","last_peered":"2024-09-13T04:58:56.634925+0000","last_clean":"2024-09-13T04:58:56.634925+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:58:56.634925+0000","last_undegraded":"2024-09-13T04:58:56.634925+0000","last_fullsized":"2024-09-13T04:58:56.634925+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":10807072,"num_objects":42,"num_object_clones":5,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":1336,"num_read_kb":3918,"num_write":520,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.972229+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:58:56.972229+0000","last_peered":"2024-09-13T04:58:56.972229+0000","last_clean":"2024-09-13T04:58:56.972229+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:58:56.972229+0000","last_undegraded":"2024-09-13T04:58:56.972229+0000","last_fullsized":"2024-09-13T04:58:56.972229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"63'164","reported_seq":3981,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.647402+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:58:57.647402+0000","last_peered":"2024-09-13T04:58:57.647402+0000","last_clean":"2024-09-13T04:58:57.647402+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:58:57.647402+0000","last_undegraded":"2024-09-13T04:58:57.647402+0000","last_fullsized":"2024-09-13T04:58:57.647402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":39021436,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":4162,"num_read_kb":20849,"num_write":156,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939347+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:58:56.939347+0000","last_peered":"2024-09-13T04:58:56.939347+0000","last_clean":"2024-09-13T04:58:56.939347+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:58:56.939347+0000","last_undegraded":"2024-09-13T04:58:56.939347+0000","last_fullsized":"2024-09-13T04:58:56.939347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"63'215","reported_seq":1295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.114774+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:58:57.114774+0000","last_peered":"2024-09-13T04:58:57.114774+0000","last_clean":"2024-09-13T04:58:57.114774+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:58:57.114774+0000","last_undegraded":"2024-09-13T04:58:57.114774+0000","last_fullsized":"2024-09-13T04:58:57.114774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":16589690,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":2112,"num_read_kb":16086,"num_write":195,"num_write_kb":6276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":58,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.939646+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:58:56.939646+0000","last_peered":"2024-09-13T04:58:56.939646+0000","last_clean":"2024-09-13T04:58:56.939646+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:58:56.939646+0000","last_undegraded":"2024-09-13T04:58:56.939646+0000","last_fullsized":"2024-09-13T04:58:56.939646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":204,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.942761+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:58:56.942761+0000","last_peered":"2024-09-13T04:58:56.942761+0000","last_clean":"2024-09-13T04:58:56.942761+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:58:56.942761+0000","last_undegraded":"2024-09-13T04:58:56.942761+0000","last_fullsized":"2024-09-13T04:58:56.942761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"63'132","reported_seq":724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.430002+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:58:57.430002+0000","last_peered":"2024-09-13T04:58:57.430002+0000","last_clean":"2024-09-13T04:58:57.430002+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:58:57.430002+0000","last_undegraded":"2024-09-13T04:58:57.430002+0000","last_fullsized":"2024-09-13T04:58:57.430002+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":5659465,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":1075,"num_read_kb":2545,"num_write":111,"num_write_kb":1540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"63'388","reported_seq":5068,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.831063+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:58:57.831063+0000","last_peered":"2024-09-13T04:58:57.831063+0000","last_clean":"2024-09-13T04:58:57.831063+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:58:57.831063+0000","last_undegraded":"2024-09-13T04:58:57.831063+0000","last_fullsized":"2024-09-13T04:58:57.831063+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":19561835,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":6175,"num_read_kb":20265,"num_write":481,"num_write_kb":9342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"63'191","reported_seq":4531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.805660+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:58:57.805660+0000","last_peered":"2024-09-13T04:58:57.805660+0000","last_clean":"2024-09-13T04:58:57.805660+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:58:57.805660+0000","last_undegraded":"2024-09-13T04:58:57.805660+0000","last_fullsized":"2024-09-13T04:58:57.805660+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":16481506,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":4529,"num_read_kb":17046,"num_write":206,"num_write_kb":11156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"63'153","reported_seq":549,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.736521+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:58:57.736521+0000","last_peered":"2024-09-13T04:58:57.736521+0000","last_clean":"2024-09-13T04:58:57.736521+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:58:57.736521+0000","last_undegraded":"2024-09-13T04:58:57.736521+0000","last_fullsized":"2024-09-13T04:58:57.736521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":28294364,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":24,"num_read":411,"num_read_kb":11466,"num_write":146,"num_write_kb":8239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"63'408","reported_seq":829,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.512564+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:58:57.512564+0000","last_peered":"2024-09-13T04:58:57.512564+0000","last_clean":"2024-09-13T04:58:57.512564+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:58:57.512564+0000","last_undegraded":"2024-09-13T04:58:57.512564+0000","last_fullsized":"2024-09-13T04:58:57.512564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":13567710,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":1839,"num_read_kb":6279,"num_write":622,"num_write_kb":4038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"63'225","reported_seq":3403,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.562423+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:58:57.562423+0000","last_peered":"2024-09-13T04:58:57.562423+0000","last_clean":"2024-09-13T04:58:57.562423+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:58:57.562423+0000","last_undegraded":"2024-09-13T04:58:57.562423+0000","last_fullsized":"2024-09-13T04:58:57.562423+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":28011766,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":4390,"num_read_kb":28951,"num_write":385,"num_write_kb":9503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"63'158","reported_seq":602,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.822983+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:58:57.822983+0000","last_peered":"2024-09-13T04:58:57.822983+0000","last_clean":"2024-09-13T04:58:57.822983+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:58:57.822983+0000","last_undegraded":"2024-09-13T04:58:57.822983+0000","last_fullsized":"2024-09-13T04:58:57.822983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":18062207,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":21,"num_read":753,"num_read_kb":6498,"num_write":141,"num_write_kb":2881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"63'438","reported_seq":5389,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.758533+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:58:57.758533+0000","last_peered":"2024-09-13T04:58:57.758533+0000","last_clean":"2024-09-13T04:58:57.758533+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:58:57.758533+0000","last_undegraded":"2024-09-13T04:58:57.758533+0000","last_fullsized":"2024-09-13T04:58:57.758533+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":20263066,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":30,"num_read":6458,"num_read_kb":20158,"num_write":653,"num_write_kb":11669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"63'226","reported_seq":4274,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:56.997077+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:58:56.997077+0000","last_peered":"2024-09-13T04:58:56.997077+0000","last_clean":"2024-09-13T04:58:56.997077+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:58:56.997077+0000","last_undegraded":"2024-09-13T04:58:56.997077+0000","last_fullsized":"2024-09-13T04:58:56.997077+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":14841877,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":36,"num_read":4773,"num_read_kb":6258,"num_write":214,"num_write_kb":891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"63'219","reported_seq":3895,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.849837+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:58:57.849837+0000","last_peered":"2024-09-13T04:58:57.849837+0000","last_clean":"2024-09-13T04:58:57.849837+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:58:57.849837+0000","last_undegraded":"2024-09-13T04:58:57.849837+0000","last_fullsized":"2024-09-13T04:58:57.849837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":43920794,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":23,"num_read":4139,"num_read_kb":26995,"num_write":229,"num_write_kb":15429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'178","reported_seq":1905,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.822484+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:58:57.822484+0000","last_peered":"2024-09-13T04:58:57.822484+0000","last_clean":"2024-09-13T04:58:57.822484+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:58:57.822484+0000","last_undegraded":"2024-09-13T04:58:57.822484+0000","last_fullsized":"2024-09-13T04:58:57.822484+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":12813357,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":1835,"num_read_kb":6905,"num_write":194,"num_write_kb":5454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"63'428","reported_seq":1181,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T04:58:57.313035+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:58:57.313035+0000","last_peered":"2024-09-13T04:58:57.313035+0000","last_clean":"2024-09-13T04:58:57.313035+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:58:57.313035+0000","last_undegraded":"2024-09-13T04:58:57.313035+0000","last_fullsized":"2024-09-13T04:58:57.313035+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":10770561,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":2735,"num_read_kb":8239,"num_write":603,"num_write_kb":4197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":665664752,"num_objects":1468,"num_object_clones":340,"num_object_copies":2936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1468,"num_whiteouts":763,"num_read":99575,"num_read_kb":462643,"num_write":9937,"num_write_kb":220319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39641088,"data_stored":327859653,"data_compressed":9555545,"data_compressed_allocated":30781440,"data_compressed_original":321218612,"omap_allocated":0,"internal_metadata":0},"log_size":8130,"ondisk_log_size":8130,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2752512,"data_stored":15301926,"data_compressed":514944,"data_compressed_allocated":1458176,"data_compressed_original":14572157,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.75900000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.876}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.044},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.0720000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.1890000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.117}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.258}]}]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40892,"kb_used_data":5680,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330948,"statfs":{"total":96636764160,"available":96594890752,"internally_reserved":0,"allocated":5816320,"data_stored":43980367,"data_compressed":1096454,"data_compressed_allocated":3846144,"data_compressed_original":42739025,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.022}]},{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.878},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.224}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.1879999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41084,"kb_used_data":5868,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330756,"statfs":{"total":96636764160,"available":96594694144,"internally_reserved":0,"allocated":6008832,"data_stored":45424984,"data_compressed":1469043,"data_compressed_allocated":4407296,"data_compressed_original":44463476,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.79200000000000004}]},{"osd":3,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":11.266,"5min":11.266,"15min":11.266},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.0169999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.76000000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50508,"kb_used_data":7100,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94321332,"statfs":{"total":96636764160,"available":96585043968,"internally_reserved":0,"allocated":7270400,"data_stored":54419661,"data_compressed":1376219,"data_compressed_allocated":5148672,"data_compressed_original":53018427,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.47},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.552}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.583}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.6180000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.7170000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":2.1499999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.9550000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":2.2229999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.879}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.796},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":2.0099999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44452,"kb_used_data":5144,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327388,"statfs":{"total":96636764160,"available":96591245312,"internally_reserved":0,"allocated":5267456,"data_stored":41872836,"data_compressed":1226689,"data_compressed_allocated":3555328,"data_compressed_original":40823847,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.378}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.23400000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.48299999999999998}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.503},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.54800000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46956,"kb_used_data":7648,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94324884,"statfs":{"total":96636764160,"available":96588681216,"internally_reserved":0,"allocated":7831552,"data_stored":57081868,"data_compressed":1503531,"data_compressed_allocated":5767168,"data_compressed_original":55773714,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.008},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.103}]},{"osd":6,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.92400000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.96499999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49028,"kb_used_data":5620,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94322812,"statfs":{"total":96636764160,"available":96586559488,"internally_reserved":0,"allocated":5754880,"data_stored":42934214,"data_compressed":1288933,"data_compressed_allocated":3780608,"data_compressed_original":41640796,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":15.755000000000001,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":15.802,"5min":15.802,"15min":15.802},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1,"5min":1,"15min":1},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":1.794,"15min":1.794},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":15.859999999999999,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":11.474,"5min":11.474,"15min":11.474},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":11.528,"5min":11.528,"15min":11.528},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.897,"5min":1.897,"15min":1.897},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":15.885999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":1.012}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":15.834,"5min":15.834,"15min":15.834},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.95899999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39796,"kb_used_data":4576,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332044,"statfs":{"total":96636764160,"available":96596013056,"internally_reserved":0,"allocated":4685824,"data_stored":30204764,"data_compressed":1034717,"data_compressed_allocated":2793472,"data_compressed_original":28998178,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.84699999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":1.1519999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.633,"5min":4.633,"15min":4.633},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":1.3089999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.6989999999999998,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.874}]},{"osd":7,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.46200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5046272,"data_stored":42601699,"data_compressed":1294200,"data_compressed_allocated":3780608,"data_compressed_original":41620316,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7024640,"data_stored":56120020,"data_compressed":1481249,"data_compressed_allocated":5709824,"data_compressed_original":55151122,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4567040,"data_stored":41617305,"data_compressed":1226987,"data_compressed_allocated":3563520,"data_compressed_original":40881191,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6586368,"data_stored":54184160,"data_compressed":1393018,"data_compressed_allocated":5181440,"data_compressed_original":53104443,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5308416,"data_stored":45140146,"data_compressed":1485269,"data_compressed_allocated":4423680,"data_compressed_original":44488052,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3932160,"data_stored":29431857,"data_compressed":1040978,"data_compressed_allocated":2772992,"data_compressed_original":28555810,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5169152,"data_stored":43782697,"data_compressed":1118706,"data_compressed_allocated":3887104,"data_compressed_original":42833233,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2007040,"data_stored":14981769,"data_compressed":515138,"data_compressed_allocated":1462272,"data_compressed_original":14584445,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:59:00.804 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T04:59:00.804 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.805 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:00.805 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T04:59:00.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T04:59:00.807 INFO:teuthology.orchestra.run.smithi001.stdout:3339 trunc from 0xa570358 to 0xe57d497 2024-09-13T04:59:00.814 INFO:teuthology.orchestra.run.smithi001.stdout:3340 write 0x6749e4f thru 0x6757791 (0xd943 bytes) 2024-09-13T04:59:00.822 INFO:teuthology.orchestra.run.smithi001.stdout:3341 writesame 0x49c1c94 thru 0x49cb224 data_size 0x1(0x9591 bytes) 2024-09-13T04:59:00.850 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:00.850 INFO:teuthology.orchestra.run.smithi001.stdout:3343 clone 23 order 21 su 8192 sc 13 2024-09-13T04:59:00.855 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:00.855 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:00.869 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:00.872 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:00.867+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:00.967 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:59:00.974 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:00.971+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:00.999 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:00.999 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:01.014 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:01.082 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone22 from 0xe57d497 (overlap 0x117536) to 0x37dd7 2024-09-13T04:59:01.085 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:01.083+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:01.152 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:59:01.152 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 67, 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "8850da77-4384-4335-9559-af2e805c4f72", 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-13T04:57:34.164121+0000", 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-13T04:59:00.958676+0000", 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-13T04:57:47.098606+0000", 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-13T04:57:36.815703+0000", 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5799936, 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-13T04:59:01.153 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "reef", 2024-09-13T04:59:01.154 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:48.984599+0000", 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:59:01.155 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:59:01.156 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "19", 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:59:01.157 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:59:01.158 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:59:01.159 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-13T04:59:01.160 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:52.756781+0000", 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:59:01.161 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-13T04:59:01.162 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "23", 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:59:01.163 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 3, 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 23, 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:59:01.164 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:59:01.165 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 4.9800000190734863, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 4.9800000190734863, 2024-09-13T04:59:01.166 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 4.380000114440918, 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 4.380000114440918, 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:58:02.446076+0000", 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:59:01.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-13T04:59:01.168 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "67", 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:59:01.169 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 26, 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 67, 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:59:01.170 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:59:01.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-13T04:59:01.172 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-13T04:59:01.173 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-13T04:59:01.173 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-13T04:59:01.178 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:59:01.178 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:59:01.178 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:59:01.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.178 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "2fda29ca-3c6d-47ae-9244-8ccdf55f89f6", 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:01.179 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:01.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.181 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.182 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.183 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6820/2346968854", 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6824/2346968854", 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/2346968854", 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/2346968854", 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9ca6e867-faf0-4295-b8eb-4c738dd26cb9", 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:01.185 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:01.186 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:01.187 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.188 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/466532899", 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6821/466532899", 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/466532899", 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/466532899", 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d8159400-8662-4567-8714-b48c2d8f3ad2", 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:01.190 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-13T04:59:01.191 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-13T04:59:01.192 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.193 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.195 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-13T04:59:01.195 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:01.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.195 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-13T04:59:01.195 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:01.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/343712578", 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/343712578", 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/343712578", 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/343712578", 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.196 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "13a3ea70-fa37-41d8-bae7-0203952b02cb", 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:01.197 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.198 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.199 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:01.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:01.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/2021827348", 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/2021827348", 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/2021827348", 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/2021827348", 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.202 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e896e6b5-c54a-405f-ab69-2a5b7b7a153e", 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6800", 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6801", 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6802", 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6803", 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6806", 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6807", 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6804", 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6805", 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6801/2183053562", 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6803/2183053562", 2024-09-13T04:59:01.207 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6807/2183053562", 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6805/2183053562", 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7de23072-7eda-43d6-9b92-d608ef82c6fa", 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:01.208 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6816", 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6817", 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6819", 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6821", 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6826", 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6828", 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6822", 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6824", 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6817/2765053006", 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6821/2765053006", 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6828/2765053006", 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6824/2765053006", 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:01.213 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8719fad9-81a0-46be-a546-db83bcace33b", 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 36, 2024-09-13T04:59:01.214 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6818", 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6820", 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6823", 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6825", 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6830", 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6831", 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.217 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6827", 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6829", 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.218 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6820/2721913793", 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6825/2721913793", 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6831/2721913793", 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6829/2721913793", 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:59:01.219 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8bf0a351-e58e-4393-a6ce-fe70acf362aa", 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6808", 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6809", 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.221 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.222 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:01.222 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6810", 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6811", 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.224 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:01.224 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.224 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6814", 2024-09-13T04:59:01.224 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:01.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6815", 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6812", 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6813", 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6809/4202513770", 2024-09-13T04:59:01.226 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6811/4202513770", 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6815/4202513770", 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6813/4202513770", 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:01.227 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.092681+0000", 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:01.228 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.414226+0000", 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.368701+0000", 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:01.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.370707+0000", 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:01.230 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.980085+0000", 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:01.231 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.874193+0000", 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.219724+0000", 2024-09-13T04:59:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:01.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.233 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:59:01.233 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:01.233 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:01.233 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:01.233 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.114987+0000", 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [ 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.6", 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:59:01.234 INFO:teuthology.orchestra.run.smithi001.stdout: 6, 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: 4 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-13T04:59:01.235 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-13T04:59:01.236 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:01.237 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:01.239 INFO:tasks.thrashosds.thrasher:j is {'epoch': 67, 'fsid': '8850da77-4384-4335-9559-af2e805c4f72', 'created': '2024-09-13T04:57:34.164121+0000', 'modified': '2024-09-13T04:59:00.958676+0000', 'last_up_change': '2024-09-13T04:57:47.098606+0000', 'last_in_change': '2024-09-13T04:57:36.815703+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T04:57:48.984599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T04:57:52.756781+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T04:58:02.446076+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 67, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2fda29ca-3c6d-47ae-9244-8ccdf55f89f6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6820', 'nonce': 2346968854}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6824', 'nonce': 2346968854}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 2346968854}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 2346968854}]}, 'public_addr': '172.21.15.1:6820/2346968854', 'cluster_addr': '172.21.15.1:6824/2346968854', 'heartbeat_back_addr': '172.21.15.1:6831/2346968854', 'heartbeat_front_addr': '172.21.15.1:6828/2346968854', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9ca6e867-faf0-4295-b8eb-4c738dd26cb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 466532899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6819', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 466532899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 466532899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6823', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 466532899}]}, 'public_addr': '172.21.15.1:6817/466532899', 'cluster_addr': '172.21.15.1:6821/466532899', 'heartbeat_back_addr': '172.21.15.1:6829/466532899', 'heartbeat_front_addr': '172.21.15.1:6825/466532899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8159400-8662-4567-8714-b48c2d8f3ad2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 343712578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 343712578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 343712578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 343712578}]}, 'public_addr': '172.21.15.1:6801/343712578', 'cluster_addr': '172.21.15.1:6803/343712578', 'heartbeat_back_addr': '172.21.15.1:6807/343712578', 'heartbeat_front_addr': '172.21.15.1:6805/343712578', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '13a3ea70-fa37-41d8-bae7-0203952b02cb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 2021827348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 2021827348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 2021827348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 2021827348}]}, 'public_addr': '172.21.15.1:6809/2021827348', 'cluster_addr': '172.21.15.1:6811/2021827348', 'heartbeat_back_addr': '172.21.15.1:6815/2021827348', 'heartbeat_front_addr': '172.21.15.1:6813/2021827348', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e896e6b5-c54a-405f-ab69-2a5b7b7a153e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 2183053562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 2183053562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 2183053562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 2183053562}]}, 'public_addr': '172.21.15.144:6801/2183053562', 'cluster_addr': '172.21.15.144:6803/2183053562', 'heartbeat_back_addr': '172.21.15.144:6807/2183053562', 'heartbeat_front_addr': '172.21.15.144:6805/2183053562', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7de23072-7eda-43d6-9b92-d608ef82c6fa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6816', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2765053006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 2765053006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6826', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6828', 'nonce': 2765053006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6824', 'nonce': 2765053006}]}, 'public_addr': '172.21.15.144:6817/2765053006', 'cluster_addr': '172.21.15.144:6821/2765053006', 'heartbeat_back_addr': '172.21.15.144:6828/2765053006', 'heartbeat_front_addr': '172.21.15.144:6824/2765053006', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8719fad9-81a0-46be-a546-db83bcace33b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2721913793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6823', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6825', 'nonce': 2721913793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6830', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6831', 'nonce': 2721913793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6827', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6829', 'nonce': 2721913793}]}, 'public_addr': '172.21.15.144:6820/2721913793', 'cluster_addr': '172.21.15.144:6825/2721913793', 'heartbeat_back_addr': '172.21.15.144:6831/2721913793', 'heartbeat_front_addr': '172.21.15.144:6829/2721913793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8bf0a351-e58e-4393-a6ce-fe70acf362aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6809', 'nonce': 4202513770}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6810', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 4202513770}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6815', 'nonce': 4202513770}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 4202513770}]}, 'public_addr': '172.21.15.144:6809/4202513770', 'cluster_addr': '172.21.15.144:6811/4202513770', 'heartbeat_back_addr': '172.21.15.144:6815/4202513770', 'heartbeat_front_addr': '172.21.15.144:6813/4202513770', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.092681+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.414226+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.368701+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.370707+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.980085+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.874193+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.219724+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.114987+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.6', 'osds': [6, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T04:59:01.239 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-13T04:59:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:01.241 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-13T04:59:01.242 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:01.235+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:01.330 INFO:teuthology.orchestra.run.smithi001.stdout:3344 write 0x5b99afe thru 0x5ba1cbb (0x81be bytes) 2024-09-13T04:59:01.339 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:01.335+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:01.354 INFO:teuthology.orchestra.run.smithi001.stdout:3347 read 0xbabef79 thru 0xbac9378 (0xa400 bytes) 2024-09-13T04:59:01.354 INFO:teuthology.orchestra.run.smithi001.stdout:3348 writesame 0x5c5ffd3 thru 0x5c6e5a3 data_size 0x9(0xe5d1 bytes) 2024-09-13T04:59:01.374 INFO:teuthology.orchestra.run.smithi001.stdout:3349 read 0xd8c1d92 thru 0xd8c4613 (0x2882 bytes) 2024-09-13T04:59:01.374 INFO:teuthology.orchestra.run.smithi001.stdout:3350 read 0x8a31642 thru 0x8a3e11e (0xcadd bytes) 2024-09-13T04:59:01.374 INFO:teuthology.orchestra.run.smithi001.stdout:3352 write 0x6432d13 thru 0x6439836 (0x6b24 bytes) 2024-09-13T04:59:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:3353 read 0x4f6c652 thru 0x4f741a0 (0x7b4f bytes) 2024-09-13T04:59:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:3354 read 0xafa449f thru 0xafa7e01 (0x3963 bytes) 2024-09-13T04:59:01.382 INFO:teuthology.orchestra.run.smithi001.stdout:3355 write 0x6116f8b thru 0x6118aee (0x1b64 bytes) 2024-09-13T04:59:01.394 INFO:teuthology.orchestra.run.smithi001.stdout:3356 trunc from 0xe57d497 to 0x6058e21 2024-09-13T04:59:01.421 INFO:teuthology.orchestra.run.smithi001.stdout:3359 read 0x2d916ac thru 0x2d9492e (0x3283 bytes) 2024-09-13T04:59:01.421 INFO:teuthology.orchestra.run.smithi001.stdout:3360 punch from 0x1536278 to 0x1538731, (0x24b9 bytes) 2024-09-13T04:59:01.421 INFO:teuthology.orchestra.run.smithi001.stdout:3361 punch from 0x2a0facc to 0x2a1082f, (0xd63 bytes) 2024-09-13T04:59:01.422 INFO:teuthology.orchestra.run.smithi001.stdout:3362 punch from 0x40e78a4 to 0x40f430e, (0xca6a bytes) 2024-09-13T04:59:01.422 INFO:teuthology.orchestra.run.smithi001.stdout:3366 write 0x46145ad thru 0x4620c80 (0xc6d4 bytes) 2024-09-13T04:59:01.425 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:01.426 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:01.441 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:01.435+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:01.444 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:01.449 INFO:teuthology.orchestra.run.smithi001.stdout:3368 punch from 0x349eb18 to 0x34a658c, (0x7a74 bytes) 2024-09-13T04:59:01.449 INFO:teuthology.orchestra.run.smithi001.stdout:3369 read 0xba19b3 thru 0xba8561 (0x6baf bytes) 2024-09-13T04:59:01.449 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.449 INFO:teuthology.orchestra.run.smithi001.stdout:3374 read 0x3cc0b6e thru 0x3cc16a4 (0xb37 bytes) 2024-09-13T04:59:01.480 INFO:teuthology.orchestra.run.smithi001.stdout:3376 write 0xe5a0ae9 thru 0xe5a828f (0x77a7 bytes) 2024-09-13T04:59:01.496 INFO:teuthology.orchestra.run.smithi001.stdout:3377 punch from 0x6bed69e to 0x6bf7d17, (0xa679 bytes) 2024-09-13T04:59:01.496 INFO:teuthology.orchestra.run.smithi001.stdout:3378 read 0xd78173c thru 0xd788710 (0x6fd5 bytes) 2024-09-13T04:59:01.498 INFO:teuthology.orchestra.run.smithi001.stdout:3379 trunc from 0xe5a8290 to 0xd693f8d 2024-09-13T04:59:01.515 INFO:teuthology.orchestra.run.smithi001.stdout:3380 punch from 0x22f2af4 to 0x22fa2c0, (0x77cc bytes) 2024-09-13T04:59:01.515 INFO:teuthology.orchestra.run.smithi001.stdout:3381 read 0x312ae56 thru 0x3131689 (0x6834 bytes) 2024-09-13T04:59:01.515 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.515 INFO:teuthology.orchestra.run.smithi001.stdout:3384 writesame 0x2153417 thru 0x215a8d4 data_size 0x33(0x74be bytes) 2024-09-13T04:59:01.532 INFO:teuthology.orchestra.run.smithi001.stdout:3385 write 0x6c70e4f thru 0x6c7fa49 (0xebfb bytes) 2024-09-13T04:59:01.541 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:01.539+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:01.554 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.572 INFO:teuthology.orchestra.run.smithi001.stdout:3388 trunc from 0xd693f8d to 0x5c74da2 2024-09-13T04:59:01.579 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:01.579 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:01.598 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:01.612 INFO:teuthology.orchestra.run.smithi001.stdout:3390 trunc from 0x5c74da2 to 0x3ab7f98 2024-09-13T04:59:01.657 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.658 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:01.655+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:01.684 INFO:teuthology.orchestra.run.smithi001.stdout:3392 write 0xccdc9a9 thru 0xccec008 (0xf660 bytes) 2024-09-13T04:59:01.708 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.708 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.708 INFO:teuthology.orchestra.run.smithi001.stdout:3396 write 0x8890f7 thru 0x88c017 (0x2f21 bytes) 2024-09-13T04:59:01.735 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.736 INFO:teuthology.orchestra.run.smithi001.stdout:3398 read 0x6b0ffde thru 0x6b18e40 (0x8e63 bytes) 2024-09-13T04:59:01.736 INFO:teuthology.orchestra.run.smithi001.stdout:3399 write 0xb1b4b42 thru 0xb1b5ebb (0x137a bytes) 2024-09-13T04:59:01.742 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:01.739+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:01.742 INFO:teuthology.orchestra.run.smithi001.stdout:3400 read 0xb7520de thru 0xb752a78 (0x99b bytes) 2024-09-13T04:59:01.742 INFO:teuthology.orchestra.run.smithi001.stdout:3402 write 0x29a6bca thru 0x29adae8 (0x6f1f bytes) 2024-09-13T04:59:01.747 INFO:teuthology.orchestra.run.smithi001.stdout:3403 read 0xc72cd44 thru 0xc738b51 (0xbe0e bytes) 2024-09-13T04:59:01.748 INFO:teuthology.orchestra.run.smithi001.stdout:3404 read 0x6a5e76d thru 0x6a6a044 (0xb8d8 bytes) 2024-09-13T04:59:01.749 INFO:teuthology.orchestra.run.smithi001.stdout:3405 trunc from 0xccec009 to 0xcae264f 2024-09-13T04:59:01.764 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:01.765 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:01.767 INFO:teuthology.orchestra.run.smithi001.stdout:3406 read 0x7da2df thru 0x7dd34c (0x306e bytes) 2024-09-13T04:59:01.767 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.767 INFO:teuthology.orchestra.run.smithi001.stdout:3409 write 0x853d3c0 thru 0x854c6a4 (0xf2e5 bytes) 2024-09-13T04:59:01.781 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:01.789 INFO:teuthology.orchestra.run.smithi001.stdout:3410 write 0x9777440 thru 0x977a30f (0x2ed0 bytes) 2024-09-13T04:59:01.795 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.795 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:01.795 INFO:teuthology.orchestra.run.smithi001.stdout:3414 writesame 0xc074747 thru 0xc077960 data_size 0x16(0x321a bytes) 2024-09-13T04:59:01.801 INFO:teuthology.orchestra.run.smithi001.stdout:3415 punch from 0xc65db3c to 0xc660bcd, (0x3091 bytes) 2024-09-13T04:59:01.801 INFO:teuthology.orchestra.run.smithi001.stdout:3417 punch from 0xa2383d6 to 0xa24025f, (0x7e89 bytes) 2024-09-13T04:59:01.801 INFO:teuthology.orchestra.run.smithi001.stdout:3418 punch from 0x38e7daf to 0x38f4aaa, (0xccfb bytes) 2024-09-13T04:59:01.801 INFO:teuthology.orchestra.run.smithi001.stdout:3420 read 0xe145ee thru 0xe1dede (0x98f1 bytes) 2024-09-13T04:59:01.818 INFO:teuthology.orchestra.run.smithi001.stdout:3421 write 0xee27645 thru 0xee2ab97 (0x3553 bytes) 2024-09-13T04:59:01.827 INFO:teuthology.orchestra.run.smithi001.stdout:3422 flatten 2024-09-13T04:59:01.843 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:01.839+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:01.911 INFO:teuthology.orchestra.run.smithi001.stdout:3423 trunc from 0xee2ab98 to 0x118c7e9 2024-09-13T04:59:01.916 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:01.936 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:01.936 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:01.944 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:01.939+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:01.956 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:01.957 INFO:teuthology.orchestra.run.smithi001.stdout:3424 punch from 0x77847 to 0x7eeb8, (0x7671 bytes) 2024-09-13T04:59:01.957 INFO:teuthology.orchestra.run.smithi001.stdout:3425 read 0x368fb thru 0x46211 (0xf917 bytes) 2024-09-13T04:59:01.958 INFO:teuthology.orchestra.run.smithi001.stdout:3426 read 0xfee577 thru 0xff7eb6 (0x9940 bytes) 2024-09-13T04:59:01.959 INFO:teuthology.orchestra.run.smithi001.stdout:3431 punch from 0x355d7f to 0x35e21d, (0x849e bytes) 2024-09-13T04:59:01.971 INFO:teuthology.orchestra.run.smithi001.stdout:3432 write 0x16d9508 thru 0x16dcb79 (0x3672 bytes) 2024-09-13T04:59:01.986 INFO:teuthology.orchestra.run.smithi001.stdout:3434 trunc from 0x16dcb7a to 0x86de8f5 2024-09-13T04:59:01.998 INFO:teuthology.orchestra.run.smithi001.stdout:3436 writesame 0x7ecca5b thru 0x7ed77be data_size 0x89(0xad64 bytes) 2024-09-13T04:59:02.013 INFO:teuthology.orchestra.run.smithi001.stdout:3437 read 0x1d776b thru 0x1e4cd5 (0xd56b bytes) 2024-09-13T04:59:02.014 INFO:teuthology.orchestra.run.smithi001.stdout:3438 writesame 0x1dc1a39 thru 0x1dc796d data_size 0x1(0x5f35 bytes) 2024-09-13T04:59:02.045 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:02.043+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:02.047 INFO:teuthology.orchestra.run.smithi001.stdout:3440 writesame 0x8884990 thru 0x8886c4d data_size 0x1(0x22be bytes) 2024-09-13T04:59:02.047 INFO:teuthology.orchestra.run.smithi001.stdout:3441 read 0x2149845 thru 0x2153563 (0x9d1f bytes) 2024-09-13T04:59:02.048 INFO:teuthology.orchestra.run.smithi001.stdout:3442 read 0x2851852 thru 0x285dd3f (0xc4ee bytes) 2024-09-13T04:59:02.049 INFO:teuthology.orchestra.run.smithi001.stdout:3443 punch from 0x7ee6e27 to 0x7ef4686, (0xd85f bytes) 2024-09-13T04:59:02.049 INFO:teuthology.orchestra.run.smithi001.stdout:3444 write 0x77421db thru 0x7748c0e (0x6a34 bytes) 2024-09-13T04:59:02.062 INFO:teuthology.orchestra.run.smithi001.stdout:3445 trunc from 0x8886c4e to 0x9372e8e 2024-09-13T04:59:02.078 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:02.078 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:02.095 INFO:teuthology.orchestra.run.smithi001.stdout:3446 write 0xde6b2ea thru 0xde75b2e (0xa845 bytes) 2024-09-13T04:59:02.095 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:02.095 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:02.097 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:02.098 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:02.112 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.112 INFO:teuthology.orchestra.run.smithi001.stdout:3448 write 0xa2bca78 thru 0xa2c77f1 (0xad7a bytes) 2024-09-13T04:59:02.115 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:02.115 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:02.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:02.131 INFO:teuthology.orchestra.run.smithi001.stdout:3449 writesame 0xc595248 thru 0xc59c3a1 data_size 0xb(0x715a bytes) 2024-09-13T04:59:02.133 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:02.133 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:02.145 INFO:teuthology.orchestra.run.smithi001.stdout:3450 writesame 0x66565ba thru 0x6656e53 data_size 0x3(0x89a bytes) 2024-09-13T04:59:02.145 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:02.143+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:3451 write 0x5046eac thru 0x504bb4b (0x4ca0 bytes) 2024-09-13T04:59:02.157 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:02.157 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:02.161 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.162 INFO:teuthology.orchestra.run.smithi001.stdout:3453 trunc from 0xde75b2f to 0xd94f921 2024-09-13T04:59:02.176 INFO:teuthology.orchestra.run.smithi001.stdout:3454 read 0xae62a5f thru 0xae6d9be (0xaf60 bytes) 2024-09-13T04:59:02.176 INFO:teuthology.orchestra.run.smithi001.stdout:3455 write 0xad0089f thru 0xad03e39 (0x359b bytes) 2024-09-13T04:59:02.179 INFO:teuthology.orchestra.run.smithi001.stdout:3456 read 0xb8ea924 thru 0xb8f00e8 (0x57c5 bytes) 2024-09-13T04:59:02.180 INFO:teuthology.orchestra.run.smithi001.stdout:3457 read 0x4ef3c7d thru 0x4ef7f9c (0x4320 bytes) 2024-09-13T04:59:02.181 INFO:teuthology.orchestra.run.smithi001.stdout:3459 read 0x9688eb8 thru 0x968f8e8 (0x6a31 bytes) 2024-09-13T04:59:02.181 INFO:teuthology.orchestra.run.smithi001.stdout:3460 write 0x218815d thru 0x2189b3e (0x19e2 bytes) 2024-09-13T04:59:02.183 INFO:teuthology.orchestra.run.smithi001.stdout:3463 write 0x1bb1ca7 thru 0x1bbbf0f (0xa269 bytes) 2024-09-13T04:59:02.195 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:02.195 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:02.195 INFO:teuthology.orchestra.run.smithi001.stdout:3465 write 0x927cebd thru 0x928aa91 (0xdbd5 bytes) 2024-09-13T04:59:02.215 INFO:teuthology.orchestra.run.smithi001.stdout:3466 writesame 0xb05d87a thru 0xb05ebb2 data_size 0x7(0x1339 bytes) 2024-09-13T04:59:02.218 INFO:teuthology.orchestra.run.smithi001.stdout:3467 read 0x4f8f0fc thru 0x4f9386d (0x4772 bytes) 2024-09-13T04:59:02.218 INFO:teuthology.orchestra.run.smithi001.stdout:3468 write 0x2a12088 thru 0x2a20369 (0xe2e2 bytes) 2024-09-13T04:59:02.231 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:02.231 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:02.247 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:02.243+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:02.253 INFO:teuthology.orchestra.run.smithi001.stdout:3469 trunc from 0xd94f921 to 0x65b085 2024-09-13T04:59:02.268 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:02.290 INFO:teuthology.orchestra.run.smithi001.stdout:3470 read 0x2d797d thru 0x2dff81 (0x8605 bytes) 2024-09-13T04:59:02.292 INFO:teuthology.orchestra.run.smithi001.stdout:3471 read 0x17cd96 thru 0x18a032 (0xd29d bytes) 2024-09-13T04:59:02.293 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.297 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:02.297 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:02.305 INFO:teuthology.orchestra.run.smithi001.stdout:3474 write 0x2c0402a thru 0x2c10853 (0xc82a bytes) 2024-09-13T04:59:02.313 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:02.332 INFO:teuthology.orchestra.run.smithi001.stdout:3475 trunc from 0x2c10854 to 0xa49c546 2024-09-13T04:59:02.342 INFO:teuthology.orchestra.run.smithi001.stdout:3476 read 0x49e69a0 thru 0x49e813a (0x179b bytes) 2024-09-13T04:59:02.342 INFO:teuthology.orchestra.run.smithi001.stdout:3477 read 0x28f7fac thru 0x29065cb (0xe620 bytes) 2024-09-13T04:59:02.343 INFO:teuthology.orchestra.run.smithi001.stdout:3478 write 0x9d28c69 thru 0x9d38a4c (0xfde4 bytes) 2024-09-13T04:59:02.347 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:02.343+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:02.365 INFO:teuthology.orchestra.run.smithi001.stdout:3479 read 0x397b11 thru 0x3985ee (0xade bytes) 2024-09-13T04:59:02.365 INFO:teuthology.orchestra.run.smithi001.stdout:3481 read 0x477bba2 thru 0x47865e6 (0xaa45 bytes) 2024-09-13T04:59:02.370 INFO:teuthology.orchestra.run.smithi001.stdout:3483 trunc from 0xa49c546 to 0xd86c19a 2024-09-13T04:59:02.381 INFO:teuthology.orchestra.run.smithi001.stdout:3486 punch from 0x74a2aa5 to 0x74ada9a, (0xaff5 bytes) 2024-09-13T04:59:02.381 INFO:teuthology.orchestra.run.smithi001.stdout:3487 punch from 0x900a4f5 to 0x900dc55, (0x3760 bytes) 2024-09-13T04:59:02.381 INFO:teuthology.orchestra.run.smithi001.stdout:3488 write 0x7640a9f thru 0x764ad5b (0xa2bd bytes) 2024-09-13T04:59:02.416 INFO:teuthology.orchestra.run.smithi001.stdout:3489 trunc from 0xd86c19a to 0xf92d3c 2024-09-13T04:59:02.445 INFO:teuthology.orchestra.run.smithi001.stdout:3490 read 0xdab002 thru 0xdb63c2 (0xb3c1 bytes) 2024-09-13T04:59:02.446 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.447 INFO:teuthology.orchestra.run.smithi001.stdout:3493 punch from 0x7cd047 to 0x7dca87, (0xfa40 bytes) 2024-09-13T04:59:02.448 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:02.443+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:02.463 INFO:teuthology.orchestra.run.smithi001.stdout:3497 writesame 0x3ca42d3 thru 0x3cb30b7 data_size 0x1(0xede5 bytes) 2024-09-13T04:59:02.463 INFO:teuthology.orchestra.run.smithi001.stdout:3499 read 0x18cb7a5 thru 0x18cfd28 (0x4584 bytes) 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.002966268000002, 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:59:02.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:59:02.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.470 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:37637 3.11 e41/24 3:88b77d11:::rbd_data.112a66af081a.0000000000000000:head v 41'72, mlcod=41'70)", 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.108767311999998, 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027255564999999999, 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355356+0000", 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356532+0000", 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001176599 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356563+0000", 2024-09-13T04:59:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0916e-05 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356576+0000", 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3293e-05 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356700+0000", 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123779 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356746+0000", 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6242000000000002e-05 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382519+0000", 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025772856 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382614+0000", 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4315999999999996e-05 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39466 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.367819894, 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022036103000000001, 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39466 2024-09-13T04:59:02.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096315+0000", 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096326+0000", 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1596999999999999e-05 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096332+0000", 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8560000000000003e-06 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112144+0000", 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015812001999999999 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112220+0000", 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5354999999999999e-05 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.118342+0000", 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061219409999999997 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40000 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 507904~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332861168000001, 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019326534999999999, 2024-09-13T04:59:02.480 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40000 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131271+0000", 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9330000000000004e-06 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131278+0000", 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9269999999999999e-06 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131282+0000", 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8729999999999998e-06 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150497+0000", 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019214709 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150537+0000", 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0197000000000002e-05 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150591+0000", 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3352e-05 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.484 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40001 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 638976~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332838082999999, 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019422915999999998, 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40001 2024-09-13T04:59:02.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131289+0000", 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.491e-06 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131306+0000", 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6489000000000001e-05 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131311+0000", 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150619+0000", 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019307642 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150657+0000", 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8015000000000001e-05 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150710+0000", 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3109999999999998e-05 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.489 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40004 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 770048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332809320999999, 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019514348000000001, 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40004 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131318+0000", 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8159999999999998e-06 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131325+0000", 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5370000000000003e-06 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131330+0000", 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6619999999999996e-06 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150741+0000", 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019410981000000001 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150778+0000", 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7793999999999998e-05 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150830+0000", 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1974000000000001e-05 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40006 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 901120~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332791405999998, 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019616892, 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.494 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40006 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131336+0000", 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1940000000000001e-06 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131342+0000", 2024-09-13T04:59:02.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0599999999999996e-06 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131346+0000", 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5569999999999997e-06 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150858+0000", 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019512057999999999 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150897+0000", 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8903999999999999e-05 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150951+0000", 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3467999999999997e-05 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40008 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1032192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.498 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:59:02.499 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332774753999999, 2024-09-13T04:59:02.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019721992000000001, 2024-09-13T04:59:02.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40008 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131350+0000", 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131360+0000", 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5079999999999994e-06 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131367+0000", 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8210000000000004e-06 2024-09-13T04:59:02.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131371+0000", 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8860000000000003e-06 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150979+0000", 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019607655000000002 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151020+0000", 2024-09-13T04:59:02.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0921999999999999e-05 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151073+0000", 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2567e-05 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40010 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1163264~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332749149000001, 2024-09-13T04:59:02.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019817818000000001, 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40010 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131378+0000", 2024-09-13T04:59:02.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131384+0000", 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0780000000000001e-06 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131389+0000", 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5079999999999999e-06 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151101+0000", 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019712159 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151141+0000", 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0679000000000001e-05 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151194+0000", 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2589000000000001e-05 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40012 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1294336~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332732792000002, 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019922019999999999, 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40012 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.508 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131392+0000", 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131394+0000", 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.029e-06 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131400+0000", 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0560000000000001e-06 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131424+0000", 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3288000000000001e-05 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151223+0000", 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019799231 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151263+0000", 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9920999999999999e-05 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151315+0000", 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1826000000000002e-05 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40014 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1425408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332696255999998, 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020004087, 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40014 2024-09-13T04:59:02.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131432+0000", 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9780000000000001e-06 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131441+0000", 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9590000000000001e-06 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131447+0000", 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9410000000000004e-06 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151343+0000", 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019896113999999999 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151381+0000", 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7969999999999997e-05 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151433+0000", 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2543000000000002e-05 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40016 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1556480~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332643818999998, 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020070607000000001, 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40016 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131481+0000", 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131484+0000", 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.326e-06 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131492+0000", 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0139999999999998e-06 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131505+0000", 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:59:02.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151461+0000", 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01995535 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151500+0000", 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8760999999999997e-05 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151552+0000", 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2533e-05 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40018 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1687552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332613995000003, 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020160419999999998, 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40018 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131514+0000", 2024-09-13T04:59:02.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0740000000000001e-06 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131520+0000", 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4479999999999999e-06 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131526+0000", 2024-09-13T04:59:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2024-09-13T04:59:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151580+0000", 2024-09-13T04:59:02.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020054424000000001 2024-09-13T04:59:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151619+0000", 2024-09-13T04:59:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9229e-05 2024-09-13T04:59:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151672+0000", 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2493e-05 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40019 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 114688~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332595022, 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020264445999999998, 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40019 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131532+0000", 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.367e-06 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131538+0000", 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1070000000000001e-06 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131543+0000", 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3930000000000001e-06 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151700+0000", 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020157191000000001 2024-09-13T04:59:02.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151741+0000", 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0864e-05 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151795+0000", 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3896e-05 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40020 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332578120999997, 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020368440000000002, 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40020 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131549+0000", 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1110000000000002e-06 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131555+0000", 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3350000000000003e-06 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131560+0000", 2024-09-13T04:59:02.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151823+0000", 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020262782999999999 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151863+0000", 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0151000000000002e-05 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151916+0000", 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2604e-05 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40021 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1949696~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332561261999999, 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020470662000000001, 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40021 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131566+0000", 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.001e-06 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131571+0000", 2024-09-13T04:59:02.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5110000000000003e-06 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131576+0000", 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151943+0000", 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020366649000000001 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151981+0000", 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8756e-05 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152035+0000", 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3335999999999999e-05 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40023 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.541 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332538829000001, 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020567854, 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40023 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131589+0000", 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4140000000000001e-06 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131597+0000", 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5220000000000007e-06 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131613+0000", 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5628999999999999e-05 2024-09-13T04:59:02.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152063+0000", 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020450032 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152101+0000", 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7886999999999998e-05 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152154+0000", 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3739999999999997e-05 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40052 3.e 3:75db92e1:::rbd_header.112a6578f5c6:head [watch reconnect cookie 139644891391936 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.332506879999997, 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021419572000000001, 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40052 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131621+0000", 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9280000000000002e-06 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131635+0000", 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4467000000000001e-05 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131642+0000", 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.144e-06 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152171+0000", 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020529544 2024-09-13T04:59:02.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152220+0000", 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.153038+0000", 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00081775299999999995 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:49893 3.11 e48/24 3:88a8e927:::rbd_data.112ae3a54590.0000000000000005:head v 48'89, mlcod=47'87)", 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.417536076000001, 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023644946999999999, 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048077+0000", 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001487817 2024-09-13T04:59:02.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048104+0000", 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7056e-05 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048117+0000", 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2914999999999999e-05 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048200+0000", 2024-09-13T04:59:02.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3042999999999998e-05 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048231+0000", 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0834999999999999e-05 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070186+0000", 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021955283999999999 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070234+0000", 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8325000000000002e-05 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:73381 3.3 e57/24 3:c56c84c9:::rbd_data.112aaf1f27a8.0000000000000000:head v 57'247, mlcod=57'245)", 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 12.256775025, 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024031073999999999, 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:02.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207348+0000", 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:02.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209308+0000", 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019604000000000002 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209340+0000", 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1741e-05 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209355+0000", 2024-09-13T04:59:02.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5625999999999999e-05 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209470+0000", 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115047 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209517+0000", 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6595999999999999e-05 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:02.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.231241+0000", 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021723592 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.231381+0000", 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014075399999999999 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:02.559 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:02.560 INFO:teuthology.orchestra.run.smithi001.stdout:3500 write 0x96258a9 thru 0x962fee5 (0xa63d bytes) 2024-09-13T04:59:02.562 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:02.563 INFO:teuthology.orchestra.run.smithi001.stdout:3501 punch from 0x632100b to 0x6324426, (0x341b bytes) 2024-09-13T04:59:02.563 INFO:teuthology.orchestra.run.smithi001.stdout:3502 trunc from 0x962fee6 to 0x526a2df 2024-09-13T04:59:02.563 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.563 INFO:teuthology.orchestra.run.smithi001.stdout:3505 punch from 0x1539f77 to 0x1548713, (0xe79c bytes) 2024-09-13T04:59:02.563 INFO:teuthology.orchestra.run.smithi001.stdout:3506 punch from 0x31df35d to 0x31e28b9, (0x355c bytes) 2024-09-13T04:59:02.564 INFO:teuthology.orchestra.run.smithi001.stdout:3507 punch from 0x15cc9e5 to 0x15ccbfa, (0x215 bytes) 2024-09-13T04:59:02.564 INFO:teuthology.orchestra.run.smithi001.stdout:3509 trunc from 0x526a2df to 0xe9359da 2024-09-13T04:59:02.564 INFO:teuthology.orchestra.run.smithi001.stdout:3511 punch from 0x71b0358 to 0x71b52ee, (0x4f96 bytes) 2024-09-13T04:59:02.564 INFO:teuthology.orchestra.run.smithi001.stdout:3512 write 0x20c127c thru 0x20c6418 (0x519d bytes) 2024-09-13T04:59:02.564 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:02.555+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:02.572 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.572 INFO:teuthology.orchestra.run.smithi001.stdout:3515 write 0xab52a19 thru 0xab61c3f (0xf227 bytes) 2024-09-13T04:59:02.597 INFO:teuthology.orchestra.run.smithi001.stdout:3516 read 0x390847c thru 0x3909495 (0x101a bytes) 2024-09-13T04:59:02.597 INFO:teuthology.orchestra.run.smithi001.stdout:3517 punch from 0x751986 to 0x75ad92, (0x940c bytes) 2024-09-13T04:59:02.597 INFO:teuthology.orchestra.run.smithi001.stdout:3518 read 0x14acd46 thru 0x14bb5f5 (0xe8b0 bytes) 2024-09-13T04:59:02.598 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.598 INFO:teuthology.orchestra.run.smithi001.stdout:3520 write 0xdce3a0b thru 0xdceb708 (0x7cfe bytes) 2024-09-13T04:59:02.617 INFO:teuthology.orchestra.run.smithi001.stdout:3521 write 0x86c2e72 thru 0x86c813a (0x52c9 bytes) 2024-09-13T04:59:02.628 INFO:teuthology.orchestra.run.smithi001.stdout:3522 read 0x13d3a1b thru 0x13e2892 (0xee78 bytes) 2024-09-13T04:59:02.629 INFO:teuthology.orchestra.run.smithi001.stdout:3523 read 0x593345f thru 0x5934a58 (0x15fa bytes) 2024-09-13T04:59:02.629 INFO:teuthology.orchestra.run.smithi001.stdout:3524 write 0x4852361 thru 0x4857fa2 (0x5c42 bytes) 2024-09-13T04:59:02.638 INFO:teuthology.orchestra.run.smithi001.stdout:3525 writesame 0x2d8f66b thru 0x2d9007c data_size 0x1(0xa12 bytes) 2024-09-13T04:59:02.643 INFO:teuthology.orchestra.run.smithi001.stdout:3527 read 0x81f6f33 thru 0x81fa077 (0x3145 bytes) 2024-09-13T04:59:02.643 INFO:teuthology.orchestra.run.smithi001.stdout:3528 write 0x3ce960 thru 0x3d5e9a (0x753b bytes) 2024-09-13T04:59:02.648 INFO:teuthology.orchestra.run.smithi001.stdout:3529 read 0x68af55f thru 0x68ba55b (0xaffd bytes) 2024-09-13T04:59:02.648 INFO:teuthology.orchestra.run.smithi001.stdout:3530 read 0x463c57b thru 0x463f77b (0x3201 bytes) 2024-09-13T04:59:02.648 INFO:teuthology.orchestra.run.smithi001.stdout:3532 read 0xcb734c8 thru 0xcb7589a (0x23d3 bytes) 2024-09-13T04:59:02.648 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.648 INFO:teuthology.orchestra.run.smithi001.stdout:3534 write 0x7d6807f thru 0x7d7647d (0xe3ff bytes) 2024-09-13T04:59:02.663 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:02.659+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:02.694 INFO:teuthology.orchestra.run.smithi001.stdout:3536 trunc from 0xe9359da to 0x182631f 2024-09-13T04:59:02.732 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:02.732 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:02.732 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:02.732 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:02.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:3537 write 0x75dd742 thru 0x75de58b (0xe4a bytes) 2024-09-13T04:59:02.763 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:02.758+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:02.767 INFO:teuthology.orchestra.run.smithi001.stdout:3538 write 0x4fc4fb2 thru 0x4fd3424 (0xe473 bytes) 2024-09-13T04:59:02.790 INFO:teuthology.orchestra.run.smithi001.stdout:3539 read 0x613b75b thru 0x613d0f3 (0x1999 bytes) 2024-09-13T04:59:02.791 INFO:teuthology.orchestra.run.smithi001.stdout:3540 read 0x2d5fcac thru 0x2d616c4 (0x1a19 bytes) 2024-09-13T04:59:02.791 INFO:teuthology.orchestra.run.smithi001.stdout:3541 read 0x37ceab thru 0x38a197 (0xd2ed bytes) 2024-09-13T04:59:02.792 INFO:teuthology.orchestra.run.smithi001.stdout:3542 punch from 0x3099612 to 0x30a2df7, (0x97e5 bytes) 2024-09-13T04:59:02.792 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.795 INFO:teuthology.orchestra.run.smithi001.stdout:3544 trunc from 0x75de58c to 0x9544c8c 2024-09-13T04:59:02.822 INFO:teuthology.orchestra.run.smithi001.stdout:3545 writesame 0xa52bb94 thru 0xa52e5cf data_size 0x35(0x2a3c bytes) 2024-09-13T04:59:02.822 INFO:teuthology.orchestra.run.smithi001.stdout:3546 read 0x4c59d58 thru 0x4c59e00 (0xa9 bytes) 2024-09-13T04:59:02.822 INFO:teuthology.orchestra.run.smithi001.stdout:3547 writesame 0xa03ebdf thru 0xa0469bc data_size 0x1(0x7dde bytes) 2024-09-13T04:59:02.864 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:02.858+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:02.867 INFO:teuthology.orchestra.run.smithi001.stdout:3548 writesame 0xd92b2bf thru 0xd9361ff data_size 0xf(0xaf41 bytes) 2024-09-13T04:59:02.881 INFO:teuthology.orchestra.run.smithi001.stdout:3549 write 0xe4d61e9 thru 0xe4e60a7 (0xfebf bytes) 2024-09-13T04:59:02.888 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:02.888 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:02.888 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:02.888 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:02.888 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:02.906 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:02.906 INFO:teuthology.orchestra.run.smithi001.stdout:3553 writesame 0x68d1b80 thru 0x68d548f data_size 0x2c(0x3910 bytes) 2024-09-13T04:59:02.907 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:02.915 INFO:teuthology.orchestra.run.smithi001.stdout:3555 read 0x750ca8a thru 0x7511ecd (0x5444 bytes) 2024-09-13T04:59:02.933 INFO:teuthology.orchestra.run.smithi001.stdout:3556 trunc from 0xe4e60a8 to 0x3ef99a0 2024-09-13T04:59:02.964 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:02.959+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:02.987 INFO:teuthology.orchestra.run.smithi001.stdout:3559 write 0xa025b0f thru 0xa0294b1 (0x39a3 bytes) 2024-09-13T04:59:02.992 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:59:03.008 INFO:teuthology.orchestra.run.smithi001.stdout:3560 write 0xa45b453 thru 0xa463a5d (0x860b bytes) 2024-09-13T04:59:03.025 INFO:teuthology.orchestra.run.smithi001.stdout:3561 read 0x86603f8 thru 0x8666491 (0x609a bytes) 2024-09-13T04:59:03.025 INFO:teuthology.orchestra.run.smithi001.stdout:3563 read 0xa02f2d3 thru 0xa03de27 (0xeb55 bytes) 2024-09-13T04:59:03.025 INFO:teuthology.orchestra.run.smithi001.stdout:3564 punch from 0x56bd283 to 0x56cbd4f, (0xeacc bytes) 2024-09-13T04:59:03.025 INFO:teuthology.orchestra.run.smithi001.stdout:3565 write 0x69c8769 thru 0x69d1375 (0x8c0d bytes) 2024-09-13T04:59:03.040 INFO:teuthology.orchestra.run.smithi001.stdout:3569 write 0x7572d6 thru 0x763b3e (0xc869 bytes) 2024-09-13T04:59:03.046 INFO:teuthology.orchestra.run.smithi001.stdout:3571 write 0x61fe82d thru 0x6206532 (0x7d06 bytes) 2024-09-13T04:59:03.061 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:03.061 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.896326591999994, 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:59:03.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:59:03.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:59:03.066 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.893885933, 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.070 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.604653638999999, 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:59:03.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:59:03.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.076 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.047053992000002, 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:59:03.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:59:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.032542939999999, 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.083 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:59:03.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70501 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 901120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793494945999999, 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048712395999999998, 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70501 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.089 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266809+0000", 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266812+0000", 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3970000000000001e-06 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266816+0000", 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.729e-06 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266820+0000", 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0690000000000003e-06 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315355+0000", 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048534985000000003 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315417+0000", 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1765000000000002e-05 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315522+0000", 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105142 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70502 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 905216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.79345921, 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048974827999999998, 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70502 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266845+0000", 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266851+0000", 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266857+0000", 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7180000000000001e-06 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266861+0000", 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6569999999999996e-06 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315576+0000", 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048714396 2024-09-13T04:59:03.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315687+0000", 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111118 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315821+0000", 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013375400000000001 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70503 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 860160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793406076, 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049096338000000003, 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70503 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266898+0000", 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266901+0000", 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266906+0000", 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4600000000000002e-06 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266911+0000", 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7570000000000004e-06 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315859+0000", 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048947955000000001 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315920+0000", 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0906000000000003e-05 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315995+0000", 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5185000000000006e-05 2024-09-13T04:59:03.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70504 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 864256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.79339045, 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049340255, 2024-09-13T04:59:03.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70504 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266917+0000", 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7180000000000001e-06 2024-09-13T04:59:03.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266922+0000", 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.028e-06 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266926+0000", 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4750000000000004e-06 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316035+0000", 2024-09-13T04:59:03.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049109070999999997 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316136+0000", 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010061 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316255+0000", 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118884 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70505 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 868352~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793375115, 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049486095000000001, 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70505 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266929+0000", 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266932+0000", 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.306e-06 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266936+0000", 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8230000000000002e-06 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266941+0000", 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.279e-06 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316291+0000", 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049349968000000001 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316344+0000", 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3659999999999998e-05 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316416+0000", 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1606999999999994e-05 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70506 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 856064~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793360632000001, 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049736427, 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70506 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266946+0000", 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3240000000000001e-06 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266951+0000", 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7489999999999998e-06 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266956+0000", 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9180000000000002e-06 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316460+0000", 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049503783000000003 2024-09-13T04:59:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316555+0000", 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316681+0000", 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125513 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70507 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 872448~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793330495999999, 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049868537999999997, 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70507 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266977+0000", 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8459999999999999e-06 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266982+0000", 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3179999999999997e-06 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266986+0000", 2024-09-13T04:59:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4320000000000001e-06 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316715+0000", 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049728886 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316772+0000", 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6805000000000002e-05 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316843+0000", 2024-09-13T04:59:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0928999999999998e-05 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70508 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 909312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793303303, 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050098392999999998, 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70508 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267001+0000", 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267004+0000", 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2589999999999999e-06 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267009+0000", 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4410000000000001e-06 2024-09-13T04:59:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267014+0000", 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6319999999999999e-06 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316884+0000", 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049869956 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316982+0000", 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8472000000000002e-05 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317100+0000", 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117977 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70509 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 876544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793287892, 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050242336999999998, 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70509 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267019+0000", 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1969999999999999e-06 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267024+0000", 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267028+0000", 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2169999999999996e-06 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317133+0000", 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050105246999999999 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317190+0000", 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6623000000000003e-05 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317259+0000", 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9599999999999998e-05 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70510 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 913408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793273338000001, 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050487833000000003, 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70510 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267031+0000", 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267033+0000", 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5660000000000001e-06 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267038+0000", 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7650000000000001e-06 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267042+0000", 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.233e-06 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317301+0000", 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050258540999999997 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317403+0000", 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101727 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317519+0000", 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116685 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70511 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 884736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793259178, 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050628583999999997, 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70511 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267045+0000", 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267048+0000", 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1430000000000001e-06 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267054+0000", 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7950000000000002e-06 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267059+0000", 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.318e-06 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317553+0000", 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050494202000000002 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317605+0000", 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2039000000000001e-05 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317674+0000", 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999996e-05 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70512 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 880640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793207367000001, 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050833717, 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70512 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267097+0000", 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267100+0000", 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7630000000000001e-06 2024-09-13T04:59:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267105+0000", 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2850000000000002e-06 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267109+0000", 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5070000000000002e-06 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317716+0000", 2024-09-13T04:59:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050606774 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317810+0000", 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3851000000000007e-05 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317931+0000", 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121177 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70513 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 892928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793191664, 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050973768000000003, 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70513 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267115+0000", 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2139999999999999e-06 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267122+0000", 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9639999999999997e-06 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267127+0000", 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4789999999999998e-06 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317967+0000", 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050840242000000001 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318019+0000", 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.189e-05 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318087+0000", 2024-09-13T04:59:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8312999999999995e-05 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70514 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793174591, 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051225729999999997, 2024-09-13T04:59:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70514 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267136+0000", 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1399999999999997e-06 2024-09-13T04:59:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267141+0000", 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0690000000000001e-06 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267145+0000", 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3830000000000002e-06 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318126+0000", 2024-09-13T04:59:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050980481000000001 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318237+0000", 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110851 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318356+0000", 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119499 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70515 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 897024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.793122556, 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051324634000000001, 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70515 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267184+0000", 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6529999999999998e-06 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267190+0000", 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267194+0000", 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4859999999999999e-06 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318381+0000", 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051186785999999998 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318436+0000", 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5180999999999999e-05 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318507+0000", 2024-09-13T04:59:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0753999999999994e-05 2024-09-13T04:59:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:03.157 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:03.158 INFO:teuthology.orchestra.run.smithi001.stdout:3574 write 0xee30c8b thru 0xee3e3c8 (0xd73e bytes) 2024-09-13T04:59:03.159 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:03.160 INFO:teuthology.orchestra.run.smithi001.stdout:3575 read 0xcffbf87 thru 0xcffe023 (0x209d bytes) 2024-09-13T04:59:03.160 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:03.160 INFO:teuthology.orchestra.run.smithi001.stdout:3578 write 0x72e9a06 thru 0x72f0398 (0x6993 bytes) 2024-09-13T04:59:03.160 INFO:teuthology.orchestra.run.smithi001.stdout:3581 read 0x76e914b thru 0x76f5212 (0xc0c8 bytes) 2024-09-13T04:59:03.161 INFO:teuthology.orchestra.run.smithi001.stdout:3582 write 0x99b37ff thru 0x99c15af (0xddb1 bytes) 2024-09-13T04:59:03.161 INFO:teuthology.orchestra.run.smithi001.stdout:3583 write 0x5526514 thru 0x552bf2a (0x5a17 bytes) 2024-09-13T04:59:03.161 INFO:teuthology.orchestra.run.smithi001.stdout:3584 writesame 0x928329c thru 0x928c889 data_size 0x3(0x95ee bytes) 2024-09-13T04:59:03.161 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:03.155+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:03.161 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:03.161 INFO:teuthology.orchestra.run.smithi001.stdout:3587 write 0xa027f3f thru 0xa02b27c (0x333e bytes) 2024-09-13T04:59:03.164 INFO:teuthology.orchestra.run.smithi001.stdout:3588 write 0x4fe981e thru 0x4ff38ea (0xa0cd bytes) 2024-09-13T04:59:03.169 INFO:teuthology.orchestra.run.smithi001.stdout:3590 read 0x3f92def thru 0x3f9c82b (0x9a3d bytes) 2024-09-13T04:59:03.169 INFO:teuthology.orchestra.run.smithi001.stdout:3591 writesame 0xdc348e5 thru 0xdc400b4 data_size 0x88(0xb7d0 bytes) 2024-09-13T04:59:03.186 INFO:teuthology.orchestra.run.smithi001.stdout:3592 read 0x60451d7 thru 0x6053671 (0xe49b bytes) 2024-09-13T04:59:03.187 INFO:teuthology.orchestra.run.smithi001.stdout:3593 read 0xcc423f6 thru 0xcc46699 (0x42a4 bytes) 2024-09-13T04:59:03.187 INFO:teuthology.orchestra.run.smithi001.stdout:3594 read 0xb77f5d2 thru 0xb782fde (0x3a0d bytes) 2024-09-13T04:59:03.187 INFO:teuthology.orchestra.run.smithi001.stdout:3595 clone 24 order 25 su 4194304 sc 2 2024-09-13T04:59:03.260 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:03.254+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:03.293 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:03.293 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:03.293 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:03.293 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:03.307 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:03.360 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:03.355+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:03.461 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:03.458+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:03.470 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:03.470 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:03.470 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:03.484 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:03.563 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:03.559+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7992274619999993, 2024-09-13T04:59:03.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:03.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:03.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.666 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7992005510000002, 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.667 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T04:59:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T04:59:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T04:59:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T04:59:03.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T04:59:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:59:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T04:59:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T04:59:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T04:59:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T04:59:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T04:59:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T04:59:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.675 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.676 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7954549150000005, 2024-09-13T04:59:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T04:59:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.677 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T04:59:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.678 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T04:59:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T04:59:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T04:59:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T04:59:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T04:59:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T04:59:03.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T04:59:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T04:59:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T04:59:03.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T04:59:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T04:59:03.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T04:59:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T04:59:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.686 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:03.687 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7919115899999998, 2024-09-13T04:59:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T04:59:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T04:59:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T04:59:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T04:59:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T04:59:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T04:59:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T04:59:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T04:59:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T04:59:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T04:59:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T04:59:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T04:59:03.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T04:59:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T04:59:03.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T04:59:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.698 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7918719620000001, 2024-09-13T04:59:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T04:59:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T04:59:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:03.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T04:59:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T04:59:03.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T04:59:03.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.791848302, 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.706 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T04:59:03.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T04:59:03.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7917607530000002, 2024-09-13T04:59:03.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T04:59:03.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7917080970000008, 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T04:59:03.716 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T04:59:03.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.720 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7877822059999993, 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T04:59:03.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:03.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T04:59:03.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T04:59:03.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.787756581, 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.726 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T04:59:03.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T04:59:03.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T04:59:03.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T04:59:03.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7877231869999992, 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:03.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7829332579999999, 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.736 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.740 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7795386289999993, 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.741 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7794659530000008, 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.745 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T04:59:03.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7757235920000003, 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.750 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T04:59:03.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7755506009999991, 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.755 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T04:59:03.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T04:59:03.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7717080020000004, 2024-09-13T04:59:03.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T04:59:03.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:59:03.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T04:59:03.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7318600239999995, 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:03.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:03.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T04:59:03.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:59:03.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T04:59:03.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7275308020000004, 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T04:59:03.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.7198018019999992, 2024-09-13T04:59:03.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T04:59:03.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:59:03.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:03.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:03.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T04:59:03.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T04:59:03.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.779 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:03.779 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:03.779 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:03.781 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:03.781 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:03.778+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:03.882 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:03.878+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:03.950 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:03.950 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:03.951 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:03.951 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:03.964 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:03.982 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:59:04.012 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:04.010+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:04.084 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:04.079+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:04.132 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:04.132 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:04.133 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:04.133 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:04.133 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:04.146 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:04.196 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:04.194+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:04.266 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:04.266 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:04.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.231099550000003, 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:59:04.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:59:04.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:59:04.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.231053912999997, 2024-09-13T04:59:04.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:59:04.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.231005828999997, 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.276 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:59:04.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.230909746999998, 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.280 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:59:04.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.230858376, 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:59:04.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:59:04.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.230812321999998, 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:59:04.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.230762261000002, 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:59:04.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.297 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 48.113347071, 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:59:04.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:04.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.303 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021918688, 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:59:04.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:59:04.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:59:04.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021894343, 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:59:04.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021812990000001, 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:59:04.317 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021743460000003, 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.318 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021716183999999, 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.322 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:59:04.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021617739, 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:59:04.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:59:04.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021548172000003, 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:59:04.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:04.335 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021477697999998, 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.339 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021452013000001, 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:59:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021416825000003, 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.344 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021392308999999, 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:59:04.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 43.021322972, 2024-09-13T04:59:04.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:59:04.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:04.357 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:04.358 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:04.358 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:04.359 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:04.360 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:04.354+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:04.372 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-13T04:59:04.378 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:04.378 INFO:teuthology.orchestra.run.smithi001.stdout:3597 write 0x8cb5c10 thru 0x8cc210f (0xc500 bytes) 2024-09-13T04:59:04.406 INFO:teuthology.orchestra.run.smithi001.stdout:3598 read 0x3f377c6 thru 0x3f41095 (0x98d0 bytes) 2024-09-13T04:59:04.406 INFO:teuthology.orchestra.run.smithi001.stdout:3599 writesame 0x5b72d8f thru 0x5b74353 data_size 0x1(0x15c5 bytes) 2024-09-13T04:59:04.425 INFO:teuthology.orchestra.run.smithi001.stdout:3601 read 0x86c1c1d thru 0x86c850d (0x68f1 bytes) 2024-09-13T04:59:04.427 INFO:teuthology.orchestra.run.smithi001.stdout:3603 trunc from 0xee3e3c9 to 0xe42c614 2024-09-13T04:59:04.448 INFO:teuthology.orchestra.run.smithi001.stdout:3604 write 0x25345b6 thru 0x253ac72 (0x66bd bytes) 2024-09-13T04:59:04.460 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:04.454+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:04.461 INFO:teuthology.orchestra.run.smithi001.stdout:3606 write 0x9f7da52 thru 0x9f8269d (0x4c4c bytes) 2024-09-13T04:59:04.463 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:04.463 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:04.463 INFO:teuthology.orchestra.run.smithi001.stdout:3611 read 0x5a09bc thru 0x5acae5 (0xc12a bytes) 2024-09-13T04:59:04.464 INFO:teuthology.orchestra.run.smithi001.stdout:3612 read 0xd47409 thru 0xd50d7a (0x9972 bytes) 2024-09-13T04:59:04.464 INFO:teuthology.orchestra.run.smithi001.stdout:3614 write 0x5443c55 thru 0x544c6c2 (0x8a6e bytes) 2024-09-13T04:59:04.485 INFO:teuthology.orchestra.run.smithi001.stdout:3615 clone 25 order 23 su 4194304 sc 9 2024-09-13T04:59:04.523 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:04.534 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:04.535 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:04.535 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:04.535 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:04.540 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:04.540 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:04.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:04.556 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:04.556 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:04.560 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:04.555+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:04.572 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:04.592 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:04.592 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:04.611 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:04.628 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:04.646 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:04.646 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:04.661 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:04.659+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:04.720 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:04.720 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:04.734 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:04.781 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:04.778+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:04.853 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:04.853 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:04.863 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:04.858+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:04.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:04.965 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:04.962+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:05.038 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:05.038 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:05.053 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:05.066 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:05.063+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:05.132 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone24 from 0xe42c614 (overlap 0xe42c614) to 0x16d13ce 2024-09-13T04:59:05.167 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:05.162+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:05.235 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:05.236 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:05.250 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:05.292 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:05.290+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:05.337 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-13T04:59:05.368 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:05.363+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:05.417 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:05.417 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:05.431 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:05.469 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:05.466+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:05.570 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:05.567+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:05.572 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:05.572 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:05.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:05.677 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:05.675+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:05.761 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:05.761 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:05.772 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:05.766+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:05.777 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:05.862 INFO:teuthology.orchestra.run.smithi001.stdout:3617 write 0x727336 thru 0x72aef2 (0x3bbd bytes) 2024-09-13T04:59:05.872 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:05.870+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:05.877 INFO:teuthology.orchestra.run.smithi001.stdout:3619 read 0x6b5aa49 thru 0x6b613f9 (0x69b1 bytes) 2024-09-13T04:59:05.877 INFO:teuthology.orchestra.run.smithi001.stdout:3620 read 0x653ae07 thru 0x653d8c4 (0x2abe bytes) 2024-09-13T04:59:05.878 INFO:teuthology.orchestra.run.smithi001.stdout:3622 read 0x1dd4afe thru 0x1de1f0d (0xd410 bytes) 2024-09-13T04:59:05.878 INFO:teuthology.orchestra.run.smithi001.stdout:3623 punch from 0x3ef25e6 to 0x3ef6eac, (0x48c6 bytes) 2024-09-13T04:59:05.897 INFO:teuthology.orchestra.run.smithi001.stdout:3624 trunc from 0xe42c614 to 0x1a0d5f8 2024-09-13T04:59:05.961 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:05.961 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:05.973 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:05.970+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:05.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:05.980 INFO:teuthology.orchestra.run.smithi001.stdout:3625 writesame 0xccb9fd6 thru 0xccc645c data_size 0x1(0xc487 bytes) 2024-09-13T04:59:05.980 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:05.981 INFO:teuthology.orchestra.run.smithi001.stdout:3627 read 0x33a6aaf thru 0x33a6fb5 (0x507 bytes) 2024-09-13T04:59:05.981 INFO:teuthology.orchestra.run.smithi001.stdout:3628 write 0x210802a thru 0x210a5c6 (0x259d bytes) 2024-09-13T04:59:05.985 INFO:teuthology.orchestra.run.smithi001.stdout:3630 write 0xc73203b thru 0xc739344 (0x730a bytes) 2024-09-13T04:59:05.992 INFO:teuthology.orchestra.run.smithi001.stdout:3631 read 0xc82060a thru 0xc8293df (0x8dd6 bytes) 2024-09-13T04:59:05.993 INFO:teuthology.orchestra.run.smithi001.stdout:3633 write 0x58af41 thru 0x595b31 (0xabf1 bytes) 2024-09-13T04:59:05.997 INFO:teuthology.orchestra.run.smithi001.stdout:3634 read 0xa9bd356 thru 0xa9c85bf (0xb26a bytes) 2024-09-13T04:59:05.997 INFO:teuthology.orchestra.run.smithi001.stdout:3636 writesame 0x2a5df1a thru 0x2a655c6 data_size 0x13(0x76ad bytes) 2024-09-13T04:59:06.001 INFO:teuthology.orchestra.run.smithi001.stdout:3637 write 0x121b4 thru 0x12abd (0x90a bytes) 2024-09-13T04:59:06.003 INFO:teuthology.orchestra.run.smithi001.stdout:3638 punch from 0xa16f337 to 0xa1731ff, (0x3ec8 bytes) 2024-09-13T04:59:06.003 INFO:teuthology.orchestra.run.smithi001.stdout:3639 writesame 0xacfa26f thru 0xad064de data_size 0x33(0xc270 bytes) 2024-09-13T04:59:06.008 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:59:06.010 INFO:teuthology.orchestra.run.smithi001.stdout:3640 writesame 0x49eb09a thru 0x49f427e data_size 0x11(0x91e5 bytes) 2024-09-13T04:59:06.018 INFO:teuthology.orchestra.run.smithi001.stdout:3641 read 0x3979872 thru 0x3987991 (0xe120 bytes) 2024-09-13T04:59:06.019 INFO:teuthology.orchestra.run.smithi001.stdout:3642 read 0x5a60354 thru 0x5a60516 (0x1c3 bytes) 2024-09-13T04:59:06.019 INFO:teuthology.orchestra.run.smithi001.stdout:3643 read 0x282f1fd thru 0x28386bf (0x94c3 bytes) 2024-09-13T04:59:06.029 INFO:teuthology.orchestra.run.smithi001.stdout:3644 trunc from 0xccc645d to 0x6a6db1f 2024-09-13T04:59:06.053 INFO:teuthology.orchestra.run.smithi001.stdout:3646 read 0x16ae6d8 thru 0x16b9415 (0xad3e bytes) 2024-09-13T04:59:06.053 INFO:teuthology.orchestra.run.smithi001.stdout:3647 write 0x46b8c96 thru 0x46c0750 (0x7abb bytes) 2024-09-13T04:59:06.057 INFO:teuthology.orchestra.run.smithi001.stdout:3648 read 0x4b34121 thru 0x4b3a36e (0x624e bytes) 2024-09-13T04:59:06.061 INFO:teuthology.orchestra.run.smithi001.stdout:3650 trunc from 0x6a6db1f to 0x4771d46 2024-09-13T04:59:06.075 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:06.070+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:06.085 INFO:teuthology.orchestra.run.smithi001.stdout:3651 read 0x3a75c4b thru 0x3a7ba5d (0x5e13 bytes) 2024-09-13T04:59:06.085 INFO:teuthology.orchestra.run.smithi001.stdout:3652 write 0x2a50d35 thru 0x2a541ad (0x3479 bytes) 2024-09-13T04:59:06.088 INFO:teuthology.orchestra.run.smithi001.stdout:3653 read 0x19425c thru 0x197df2 (0x3b97 bytes) 2024-09-13T04:59:06.108 INFO:teuthology.orchestra.run.smithi001.stdout:3654 write 0x8d73eab thru 0x8d7bfc4 (0x811a bytes) 2024-09-13T04:59:06.114 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:06.114 INFO:teuthology.orchestra.run.smithi001.stdout:3656 read 0x25277e5 thru 0x252b359 (0x3b75 bytes) 2024-09-13T04:59:06.115 INFO:teuthology.orchestra.run.smithi001.stdout:3658 read 0x1054ccf thru 0x105cf46 (0x8278 bytes) 2024-09-13T04:59:06.116 INFO:teuthology.orchestra.run.smithi001.stdout:3659 punch from 0x862679f to 0x8628525, (0x1d86 bytes) 2024-09-13T04:59:06.116 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:06.116 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:06.136 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:06.137 INFO:teuthology.orchestra.run.smithi001.stdout:3660 write 0xcdbf78b thru 0xcdca1eb (0xaa61 bytes) 2024-09-13T04:59:06.142 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:06.143 INFO:teuthology.orchestra.run.smithi001.stdout:3663 write 0x6871820 thru 0x6875769 (0x3f4a bytes) 2024-09-13T04:59:06.147 INFO:teuthology.orchestra.run.smithi001.stdout:3664 trunc from 0xcdca1ec to 0xc6e22aa 2024-09-13T04:59:06.159 INFO:teuthology.orchestra.run.smithi001.stdout:3666 read 0x45829da thru 0x458ca10 (0xa037 bytes) 2024-09-13T04:59:06.159 INFO:teuthology.orchestra.run.smithi001.stdout:3667 read 0x442427d thru 0x442b459 (0x71dd bytes) 2024-09-13T04:59:06.159 INFO:teuthology.orchestra.run.smithi001.stdout:3669 write 0x36cb35c thru 0x36d0118 (0x4dbd bytes) 2024-09-13T04:59:06.162 INFO:teuthology.orchestra.run.smithi001.stdout:3670 read 0xbfae8e3 thru 0xbfbcb6c (0xe28a bytes) 2024-09-13T04:59:06.162 INFO:teuthology.orchestra.run.smithi001.stdout:3671 read 0x1ded1f0 thru 0x1defaf3 (0x2904 bytes) 2024-09-13T04:59:06.165 INFO:teuthology.orchestra.run.smithi001.stdout:3672 trunc from 0xc6e22aa to 0xa54dbd9 2024-09-13T04:59:06.176 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:06.174+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:06.184 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:06.184 INFO:teuthology.orchestra.run.smithi001.stdout:3675 punch from 0xd7006 to 0xe5d21, (0xed1b bytes) 2024-09-13T04:59:06.184 INFO:teuthology.orchestra.run.smithi001.stdout:3676 read 0x5388285 thru 0x538df18 (0x5c94 bytes) 2024-09-13T04:59:06.184 INFO:teuthology.orchestra.run.smithi001.stdout:3677 write 0x837a95b thru 0x837c57e (0x1c24 bytes) 2024-09-13T04:59:06.192 INFO:teuthology.orchestra.run.smithi001.stdout:3678 trunc from 0xa54dbd9 to 0x7b3b7ef 2024-09-13T04:59:06.221 INFO:teuthology.orchestra.run.smithi001.stdout:3679 writesame 0xb89bc03 thru 0xb8a6225 data_size 0x1(0xa623 bytes) 2024-09-13T04:59:06.222 INFO:teuthology.orchestra.run.smithi001.stdout:3680 read 0x29766c0 thru 0x2981c98 (0xb5d9 bytes) 2024-09-13T04:59:06.222 INFO:teuthology.orchestra.run.smithi001.stdout:3681 read 0x4c126af thru 0x4c173f5 (0x4d47 bytes) 2024-09-13T04:59:06.228 INFO:teuthology.orchestra.run.smithi001.stdout:3682 trunc from 0xb8a6226 to 0x7711f61 2024-09-13T04:59:06.240 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:06.241 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:06.241 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T04:59:06.242 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T04:59:06.248 INFO:teuthology.orchestra.run.smithi001.stdout:3684 trunc from 0x7711f61 to 0xc7c965e 2024-09-13T04:59:06.259 INFO:teuthology.orchestra.run.smithi001.stdout:3686 write 0x356a246 thru 0x3570c2d (0x69e8 bytes) 2024-09-13T04:59:06.261 INFO:teuthology.orchestra.run.smithi001.stdout:3687 write 0x26c5af9 thru 0x26c65ec (0xaf4 bytes) 2024-09-13T04:59:06.268 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:06.268 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:06.277 INFO:teuthology.orchestra.run.smithi001.stdout:3688 writesame 0xdf6b411 thru 0xdf7a21a data_size 0x1(0xee0a bytes) 2024-09-13T04:59:06.277 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:06.277 INFO:teuthology.orchestra.run.smithi001.stdout:3692 clone 26 order 20 su 65536 sc 5 2024-09-13T04:59:06.277 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:06.274+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:06.284 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:06.378 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:06.375+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:06.448 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:06.448 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:06.462 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:06.506 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:06.503+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:06.542 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:59:06.542 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:06.542 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 69, 2024-09-13T04:59:06.542 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "8850da77-4384-4335-9559-af2e805c4f72", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-09-13T04:57:34.164121+0000", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-09-13T04:59:04.984730+0000", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2024-09-13T04:57:47.098606+0000", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2024-09-13T04:57:36.815703+0000", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5801984, 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "noscrub", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2024-09-13T04:59:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2024-09-13T04:59:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "reef", 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_crimson": false, 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": ".mgr", 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:48.984599+0000", 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:59:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:59:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "19", 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2024-09-13T04:59:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:59:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:59:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_max": 32, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": {} 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 8, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 8, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.25, 2024-09-13T04:59:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 2, 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 2, 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:57:52.756781+0000", 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:59:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:59:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "23", 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 3, 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 23, 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:59:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:59:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:59:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 4.9800000190734863, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 4.9800000190734863, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 4.380000114440918, 2024-09-13T04:59:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 4.380000114440918, 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "pool_client.0", 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2024-09-13T04:58:02.446076+0000", 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2024-09-13T04:59:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 32, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 32, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 32, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 32, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 32, 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2024-09-13T04:59:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "69", 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2024-09-13T04:59:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 28, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 69, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2024-09-13T04:59:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2024-09-13T04:59:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "read_balance": { 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "score_acting": 1.75, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "score_stable": 1.75, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "optimal_score": 1, 2024-09-13T04:59:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_acting": 1.75, 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "raw_score_stable": 1.75, 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity_weighted": 1, 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity": 1, 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "average_primary_affinity_weighted": 1 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "2fda29ca-3c6d-47ae-9244-8ccdf55f89f6", 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2024-09-13T04:59:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2024-09-13T04:59:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2346968854 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6820/2346968854", 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6824/2346968854", 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/2346968854", 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/2346968854", 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "9ca6e867-faf0-4295-b8eb-4c738dd26cb9", 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 466532899 2024-09-13T04:59:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6817/466532899", 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6821/466532899", 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/466532899", 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/466532899", 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:59:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "d8159400-8662-4567-8714-b48c2d8f3ad2", 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 343712578 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/343712578", 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/343712578", 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/343712578", 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/343712578", 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "13a3ea70-fa37-41d8-bae7-0203952b02cb", 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2024-09-13T04:59:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2021827348 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/2021827348", 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/2021827348", 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/2021827348", 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/2021827348", 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e896e6b5-c54a-405f-ab69-2a5b7b7a153e", 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6800", 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6801", 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6802", 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6803", 2024-09-13T04:59:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6806", 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6807", 2024-09-13T04:59:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6804", 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6805", 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183053562 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6801/2183053562", 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6803/2183053562", 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6807/2183053562", 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6805/2183053562", 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "7de23072-7eda-43d6-9b92-d608ef82c6fa", 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 14, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6816", 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6817", 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6819", 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6821", 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6826", 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6828", 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6822", 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6824", 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2765053006 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6817/2765053006", 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6821/2765053006", 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6828/2765053006", 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6824/2765053006", 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8719fad9-81a0-46be-a546-db83bcace33b", 2024-09-13T04:59:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 36, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6818", 2024-09-13T04:59:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6820", 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6823", 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6825", 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6830", 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6831", 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6827", 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6829", 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2721913793 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6820/2721913793", 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6825/2721913793", 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6831/2721913793", 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6829/2721913793", 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:59:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8bf0a351-e58e-4393-a6ce-fe70acf362aa", 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 15, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 24, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6808", 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6809", 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2024-09-13T04:59:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6810", 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6811", 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2024-09-13T04:59:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6814", 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6815", 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6812", 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.144:6813", 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4202513770 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.144:6809/4202513770", 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.144:6811/4202513770", 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.144:6815/4202513770", 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.144:6813/4202513770", 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.092681+0000", 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.414226+0000", 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.368701+0000", 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.370707+0000", 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.980085+0000", 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:41.874193+0000", 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.219724+0000", 2024-09-13T04:59:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138322906710015, 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2024-09-13T04:57:42.114987+0000", 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [ 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "3.6", 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: 6, 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: 4 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_primaries": [], 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2024-09-13T04:59:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "range_blocklist": {}, 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [], 2024-09-13T04:59:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.618 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:06.620 INFO:tasks.thrashosds.thrasher:j is {'epoch': 69, 'fsid': '8850da77-4384-4335-9559-af2e805c4f72', 'created': '2024-09-13T04:57:34.164121+0000', 'modified': '2024-09-13T04:59:04.984730+0000', 'last_up_change': '2024-09-13T04:57:47.098606+0000', 'last_in_change': '2024-09-13T04:57:36.815703+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T04:57:48.984599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T04:57:52.756781+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T04:58:02.446076+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2fda29ca-3c6d-47ae-9244-8ccdf55f89f6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6818', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6820', 'nonce': 2346968854}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6822', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6824', 'nonce': 2346968854}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 2346968854}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 2346968854}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 2346968854}]}, 'public_addr': '172.21.15.1:6820/2346968854', 'cluster_addr': '172.21.15.1:6824/2346968854', 'heartbeat_back_addr': '172.21.15.1:6831/2346968854', 'heartbeat_front_addr': '172.21.15.1:6828/2346968854', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9ca6e867-faf0-4295-b8eb-4c738dd26cb9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6817', 'nonce': 466532899}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6819', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6821', 'nonce': 466532899}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 466532899}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6823', 'nonce': 466532899}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 466532899}]}, 'public_addr': '172.21.15.1:6817/466532899', 'cluster_addr': '172.21.15.1:6821/466532899', 'heartbeat_back_addr': '172.21.15.1:6829/466532899', 'heartbeat_front_addr': '172.21.15.1:6825/466532899', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd8159400-8662-4567-8714-b48c2d8f3ad2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 343712578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 343712578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 343712578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 343712578}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 343712578}]}, 'public_addr': '172.21.15.1:6801/343712578', 'cluster_addr': '172.21.15.1:6803/343712578', 'heartbeat_back_addr': '172.21.15.1:6807/343712578', 'heartbeat_front_addr': '172.21.15.1:6805/343712578', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '13a3ea70-fa37-41d8-bae7-0203952b02cb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 2021827348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 2021827348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 2021827348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 2021827348}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 2021827348}]}, 'public_addr': '172.21.15.1:6809/2021827348', 'cluster_addr': '172.21.15.1:6811/2021827348', 'heartbeat_back_addr': '172.21.15.1:6815/2021827348', 'heartbeat_front_addr': '172.21.15.1:6813/2021827348', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e896e6b5-c54a-405f-ab69-2a5b7b7a153e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6800', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6801', 'nonce': 2183053562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6802', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6803', 'nonce': 2183053562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6806', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6807', 'nonce': 2183053562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6804', 'nonce': 2183053562}, {'type': 'v1', 'addr': '172.21.15.144:6805', 'nonce': 2183053562}]}, 'public_addr': '172.21.15.144:6801/2183053562', 'cluster_addr': '172.21.15.144:6803/2183053562', 'heartbeat_back_addr': '172.21.15.144:6807/2183053562', 'heartbeat_front_addr': '172.21.15.144:6805/2183053562', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7de23072-7eda-43d6-9b92-d608ef82c6fa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6816', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6817', 'nonce': 2765053006}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6819', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6821', 'nonce': 2765053006}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6826', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6828', 'nonce': 2765053006}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6822', 'nonce': 2765053006}, {'type': 'v1', 'addr': '172.21.15.144:6824', 'nonce': 2765053006}]}, 'public_addr': '172.21.15.144:6817/2765053006', 'cluster_addr': '172.21.15.144:6821/2765053006', 'heartbeat_back_addr': '172.21.15.144:6828/2765053006', 'heartbeat_front_addr': '172.21.15.144:6824/2765053006', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '8719fad9-81a0-46be-a546-db83bcace33b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6818', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6820', 'nonce': 2721913793}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6823', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6825', 'nonce': 2721913793}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6830', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6831', 'nonce': 2721913793}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6827', 'nonce': 2721913793}, {'type': 'v1', 'addr': '172.21.15.144:6829', 'nonce': 2721913793}]}, 'public_addr': '172.21.15.144:6820/2721913793', 'cluster_addr': '172.21.15.144:6825/2721913793', 'heartbeat_back_addr': '172.21.15.144:6831/2721913793', 'heartbeat_front_addr': '172.21.15.144:6829/2721913793', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8bf0a351-e58e-4393-a6ce-fe70acf362aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6808', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6809', 'nonce': 4202513770}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6810', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6811', 'nonce': 4202513770}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6814', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6815', 'nonce': 4202513770}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6812', 'nonce': 4202513770}, {'type': 'v1', 'addr': '172.21.15.144:6813', 'nonce': 4202513770}]}, 'public_addr': '172.21.15.144:6809/4202513770', 'cluster_addr': '172.21.15.144:6811/4202513770', 'heartbeat_back_addr': '172.21.15.144:6815/4202513770', 'heartbeat_front_addr': '172.21.15.144:6813/4202513770', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.092681+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.414226+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.368701+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.370707+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.980085+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:41.874193+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.219724+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T04:57:42.114987+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.6', 'osds': [6, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T04:59:06.620 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:59:06.621 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:06.615+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:06.631 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:06.631 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:06.646 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:06.665 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:06.721 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:06.718+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:06.819 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:06.819 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:06.821 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:06.819+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:06.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:06.849 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:06.878 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:06.879 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:06.896 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:06.897 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:06.913 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:06.913 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:06.923 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:06.918+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:06.934 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:06.934 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:06.956 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:06.956 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:06.969 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:06.969 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.506518661000001, 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:06.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:06.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:59:06.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:06.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:37637 3.11 e41/24 3:88b77d11:::rbd_data.112a66af081a.0000000000000000:head v 41'72, mlcod=41'70)", 2024-09-13T04:59:06.974 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.612319704999997, 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027255564999999999, 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:06.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355358+0000", 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.355356+0000", 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356532+0000", 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001176599 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356563+0000", 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0916e-05 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356576+0000", 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3293e-05 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356700+0000", 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000123779 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.356746+0000", 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6242000000000002e-05 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382519+0000", 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025772856 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:24.382614+0000", 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4315999999999996e-05 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:39466 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.871372287, 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022036103000000001, 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:06.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 39466 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:59:06.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096305+0000", 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096315+0000", 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7100000000000002e-06 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096326+0000", 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1596999999999999e-05 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.096332+0000", 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8560000000000003e-06 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112144+0000", 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015812001999999999 2024-09-13T04:59:06.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.112220+0000", 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5354999999999999e-05 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.118342+0000", 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061219409999999997 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40000 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 507904~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836413561000001, 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019326534999999999, 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40000 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:06.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131264+0000", 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:06.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131271+0000", 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9330000000000004e-06 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131278+0000", 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9269999999999999e-06 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131282+0000", 2024-09-13T04:59:06.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8729999999999998e-06 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150497+0000", 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019214709 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150537+0000", 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0197000000000002e-05 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150591+0000", 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3352e-05 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40001 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 638976~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836390475999998, 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019422915999999998, 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:06.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40001 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:59:06.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131287+0000", 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131289+0000", 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.491e-06 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131306+0000", 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6489000000000001e-05 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131311+0000", 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7379999999999999e-06 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150619+0000", 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019307642 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150657+0000", 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8015000000000001e-05 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150710+0000", 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3109999999999998e-05 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.992 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40004 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 770048~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836361713999999, 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019514348000000001, 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40004 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:06.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131316+0000", 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131318+0000", 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8159999999999998e-06 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131325+0000", 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5370000000000003e-06 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131330+0000", 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6619999999999996e-06 2024-09-13T04:59:06.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150741+0000", 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019410981000000001 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150778+0000", 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7793999999999998e-05 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150830+0000", 2024-09-13T04:59:06.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1974000000000001e-05 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40006 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 901120~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836343798999998, 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019616892, 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:06.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40006 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:06.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131334+0000", 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131336+0000", 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1940000000000001e-06 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131342+0000", 2024-09-13T04:59:06.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0599999999999996e-06 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131346+0000", 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5569999999999997e-06 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150858+0000", 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019512057999999999 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150897+0000", 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8903999999999999e-05 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150951+0000", 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3467999999999997e-05 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40008 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1032192~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:59:07.001 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836327146999999, 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019721992000000001, 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40008 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:59:07.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131351+0000", 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131350+0000", 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131360+0000", 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5079999999999994e-06 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131367+0000", 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8210000000000004e-06 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131371+0000", 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8860000000000003e-06 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.150979+0000", 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019607655000000002 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151020+0000", 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0921999999999999e-05 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151073+0000", 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2567e-05 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40010 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1163264~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836301542000001, 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019817818000000001, 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40010 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.006 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131376+0000", 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131378+0000", 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2010000000000002e-06 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131384+0000", 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0780000000000001e-06 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131389+0000", 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5079999999999999e-06 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151101+0000", 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019712159 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151141+0000", 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0679000000000001e-05 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151194+0000", 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2589000000000001e-05 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40012 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1294336~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836285185000001, 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019922019999999999, 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.010 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40012 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131393+0000", 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131392+0000", 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131394+0000", 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.029e-06 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131400+0000", 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0560000000000001e-06 2024-09-13T04:59:07.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131424+0000", 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3288000000000001e-05 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151223+0000", 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019799231 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151263+0000", 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9920999999999999e-05 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151315+0000", 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1826000000000002e-05 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40014 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1425408~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.014 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836248648999998, 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020004087, 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40014 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:59:07.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131429+0000", 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131432+0000", 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9780000000000001e-06 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131441+0000", 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9590000000000001e-06 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131447+0000", 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9410000000000004e-06 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151343+0000", 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019896113999999999 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151381+0000", 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7969999999999997e-05 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151433+0000", 2024-09-13T04:59:07.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2543000000000002e-05 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40016 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1556480~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836196211999997, 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020070607000000001, 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40016 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131482+0000", 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131481+0000", 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131484+0000", 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.326e-06 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131492+0000", 2024-09-13T04:59:07.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0139999999999998e-06 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131505+0000", 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3216e-05 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151461+0000", 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01995535 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151500+0000", 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8760999999999997e-05 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151552+0000", 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2533e-05 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40018 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1687552~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836166388000002, 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020160419999999998, 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40018 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131511+0000", 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131514+0000", 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0740000000000001e-06 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131520+0000", 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4479999999999999e-06 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131526+0000", 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2024-09-13T04:59:07.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151580+0000", 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020054424000000001 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151619+0000", 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9229e-05 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151672+0000", 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2493e-05 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40019 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 114688~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836147414999999, 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020264445999999998, 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.028 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40019 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131530+0000", 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131532+0000", 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.367e-06 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131538+0000", 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1070000000000001e-06 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131543+0000", 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3930000000000001e-06 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151700+0000", 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020157191000000001 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151741+0000", 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0864e-05 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151795+0000", 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3896e-05 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40020 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.032 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836130513999997, 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020368440000000002, 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40020 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:59:07.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131547+0000", 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131549+0000", 2024-09-13T04:59:07.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1110000000000002e-06 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131555+0000", 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3350000000000003e-06 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131560+0000", 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151823+0000", 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020262782999999999 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151863+0000", 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0151000000000002e-05 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151916+0000", 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2604e-05 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40021 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 1949696~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836113654999998, 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020470662000000001, 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40021 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.037 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131564+0000", 2024-09-13T04:59:07.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131566+0000", 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.001e-06 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131571+0000", 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5110000000000003e-06 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131576+0000", 2024-09-13T04:59:07.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7589999999999997e-06 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151943+0000", 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020366649000000001 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.151981+0000", 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8756e-05 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152035+0000", 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3335999999999999e-05 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40023 3.e 3:74cbd711:::rbd_data.112a66af081a.0000000000000004:b [read 2080768~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836091222, 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020567854, 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.041 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40023 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131586+0000", 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131589+0000", 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4140000000000001e-06 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131597+0000", 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5220000000000007e-06 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131613+0000", 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5628999999999999e-05 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152063+0000", 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020450032 2024-09-13T04:59:07.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152101+0000", 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7886999999999998e-05 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152154+0000", 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3739999999999997e-05 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.045 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40052 3.e 3:75db92e1:::rbd_header.112a6578f5c6:head [watch reconnect cookie 139644891391936 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.836059273000004, 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021419572000000001, 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40052 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131618+0000", 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131621+0000", 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9280000000000002e-06 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131635+0000", 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4467000000000001e-05 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.131642+0000", 2024-09-13T04:59:07.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.144e-06 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152171+0000", 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020529544 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.152220+0000", 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9187999999999997e-05 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:29.153038+0000", 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00081775299999999995 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:49893 3.11 e48/24 3:88a8e927:::rbd_data.112ae3a54590.0000000000000005:head v 48'89, mlcod=47'87)", 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.921088469000001, 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023644946999999999, 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.046589+0000", 2024-09-13T04:59:07.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048077+0000", 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001487817 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048104+0000", 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7056e-05 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048117+0000", 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2914999999999999e-05 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048200+0000", 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3042999999999998e-05 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.048231+0000", 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0834999999999999e-05 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070186+0000", 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021955283999999999 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:36.070234+0000", 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8325000000000002e-05 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.054 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:73381 3.3 e57/24 3:c56c84c9:::rbd_data.112aaf1f27a8.0000000000000000:head v 57'247, mlcod=57'245)", 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 16.760327417999999, 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024031073999999999, 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207350+0000", 2024-09-13T04:59:07.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.207348+0000", 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209308+0000", 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019604000000000002 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209340+0000", 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1741e-05 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209355+0000", 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5625999999999999e-05 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209470+0000", 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115047 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.209517+0000", 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6595999999999999e-05 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.231241+0000", 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021723592 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:50.231381+0000", 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014075399999999999 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.059 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.059 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:07.060 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:59:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:59:07.062 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:07.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:07.064 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-13T04:59:06.926101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":711165844,"num_objects":1594,"num_object_clones":393,"num_object_copies":3188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1594,"num_whiteouts":815,"num_read":105447,"num_read_kb":494880,"num_write":11694,"num_write_kb":234015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9508,"ondisk_log_size":9508,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":320820,"kb_used_data":47276,"kb_used_omap":12,"kb_used_meta":273395,"kb_avail":754653900,"statfs":{"total":773094113280,"available":772765593600,"internally_reserved":0,"allocated":48410624,"data_stored":346491180,"data_compressed":10802102,"data_compressed_allocated":32915456,"data_compressed_original":336876514,"omap_allocated":12743,"internal_metadata":279957049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44910705,"num_objects":122,"num_object_clones":53,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":52,"num_read":5756,"num_read_kb":32137,"num_write":1596,"num_write_kb":11278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001430"},"pg_stats":[{"pgid":"3.1f","version":"67'317","reported_seq":2287,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:01.801265+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:59:01.801265+0000","last_peered":"2024-09-13T04:59:01.801265+0000","last_clean":"2024-09-13T04:59:01.801265+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:59:01.801265+0000","last_undegraded":"2024-09-13T04:59:01.801265+0000","last_fullsized":"2024-09-13T04:59:01.801265+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":34399198,"num_objects":59,"num_object_clones":18,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":3099,"num_read_kb":12600,"num_write":368,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"67'177","reported_seq":1087,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:01.921692+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:59:01.921692+0000","last_peered":"2024-09-13T04:59:01.921692+0000","last_clean":"2024-09-13T04:59:01.921692+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:59:01.921692+0000","last_undegraded":"2024-09-13T04:59:01.921692+0000","last_fullsized":"2024-09-13T04:59:01.921692+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":5841927,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":1781,"num_read_kb":7057,"num_write":155,"num_write_kb":3203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"67'387","reported_seq":1015,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.900184+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:59:02.900184+0000","last_peered":"2024-09-13T04:59:02.900184+0000","last_clean":"2024-09-13T04:59:02.900184+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:59:02.900184+0000","last_undegraded":"2024-09-13T04:59:02.900184+0000","last_fullsized":"2024-09-13T04:59:02.900184+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":31352578,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":31,"num_read":1710,"num_read_kb":12421,"num_write":534,"num_write_kb":6509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"67'297","reported_seq":896,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.722468+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:59:02.722468+0000","last_peered":"2024-09-13T04:59:02.722468+0000","last_clean":"2024-09-13T04:59:02.722468+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:59:02.722468+0000","last_undegraded":"2024-09-13T04:59:02.722468+0000","last_fullsized":"2024-09-13T04:59:02.722468+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":10832571,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":25,"num_read":1159,"num_read_kb":6414,"num_write":345,"num_write_kb":6305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"67'318","reported_seq":4544,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.723664+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:59:02.723664+0000","last_peered":"2024-09-13T04:59:02.723664+0000","last_clean":"2024-09-13T04:59:02.723664+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:59:02.723664+0000","last_undegraded":"2024-09-13T04:59:02.723664+0000","last_fullsized":"2024-09-13T04:59:02.723664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":21591428,"num_objects":45,"num_object_clones":14,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":5270,"num_read_kb":15434,"num_write":370,"num_write_kb":5625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"67'153","reported_seq":1721,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.866774+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:59:02.866774+0000","last_peered":"2024-09-13T04:59:02.866774+0000","last_clean":"2024-09-13T04:59:02.866774+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:59:02.866774+0000","last_undegraded":"2024-09-13T04:59:02.866774+0000","last_fullsized":"2024-09-13T04:59:02.866774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":22340844,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":1657,"num_read_kb":7726,"num_write":156,"num_write_kb":5397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"67'306","reported_seq":17524,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.427612+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:59:02.427612+0000","last_peered":"2024-09-13T04:59:02.427612+0000","last_clean":"2024-09-13T04:59:02.427612+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:59:02.427612+0000","last_undegraded":"2024-09-13T04:59:02.427612+0000","last_fullsized":"2024-09-13T04:59:02.427612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":46187341,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":18215,"num_read_kb":95496,"num_write":369,"num_write_kb":18078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"67'181","reported_seq":2302,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:01.861636+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:59:01.861636+0000","last_peered":"2024-09-13T04:59:01.861636+0000","last_clean":"2024-09-13T04:59:01.861636+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:59:01.861636+0000","last_undegraded":"2024-09-13T04:59:01.861636+0000","last_fullsized":"2024-09-13T04:59:01.861636+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":24831652,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":31,"num_read":2179,"num_read_kb":6571,"num_write":159,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"67'166","reported_seq":2821,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:01.921972+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:59:01.921972+0000","last_peered":"2024-09-13T04:59:01.921972+0000","last_clean":"2024-09-13T04:59:01.921972+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:59:01.921972+0000","last_undegraded":"2024-09-13T04:59:01.921972+0000","last_fullsized":"2024-09-13T04:59:01.921972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":17007635,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":25,"num_read":2658,"num_read_kb":20419,"num_write":180,"num_write_kb":12724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"67'350","reported_seq":777,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.712500+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:59:02.712500+0000","last_peered":"2024-09-13T04:59:02.712500+0000","last_clean":"2024-09-13T04:59:02.712500+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:59:02.712500+0000","last_undegraded":"2024-09-13T04:59:02.712500+0000","last_fullsized":"2024-09-13T04:59:02.712500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":9156968,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":1437,"num_read_kb":7033,"num_write":475,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"67'280","reported_seq":755,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.791602+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:59:02.791602+0000","last_peered":"2024-09-13T04:59:02.791602+0000","last_clean":"2024-09-13T04:59:02.791602+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:59:02.791602+0000","last_undegraded":"2024-09-13T04:59:02.791602+0000","last_fullsized":"2024-09-13T04:59:02.791602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":16517343,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":1277,"num_read_kb":9204,"num_write":334,"num_write_kb":4983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"67'349","reported_seq":2944,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.643211+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:59:02.643211+0000","last_peered":"2024-09-13T04:59:02.643211+0000","last_clean":"2024-09-13T04:59:02.643211+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:59:02.643211+0000","last_undegraded":"2024-09-13T04:59:02.643211+0000","last_fullsized":"2024-09-13T04:59:02.643211+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":18704381,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":3648,"num_read_kb":11899,"num_write":445,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:59:00.004895+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:59:00.004895+0000","last_peered":"2024-09-13T04:59:00.004895+0000","last_clean":"2024-09-13T04:59:00.004895+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:59:00.004895+0000","last_undegraded":"2024-09-13T04:59:00.004895+0000","last_fullsized":"2024-09-13T04:59:00.004895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"67'180","reported_seq":2243,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.710244+0000","last_change":"2024-09-13T04:58:19.436873+0000","last_active":"2024-09-13T04:59:02.710244+0000","last_peered":"2024-09-13T04:59:02.710244+0000","last_clean":"2024-09-13T04:59:02.710244+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T04:59:02.710244+0000","last_undegraded":"2024-09-13T04:59:02.710244+0000","last_fullsized":"2024-09-13T04:59:02.710244+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":42976958,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":2106,"num_read_kb":21020,"num_write":193,"num_write_kb":17638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:58:59.961730+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:58:59.961730+0000","last_peered":"2024-09-13T04:58:59.961730+0000","last_clean":"2024-09-13T04:58:59.961730+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:58:59.961730+0000","last_undegraded":"2024-09-13T04:58:59.961730+0000","last_fullsized":"2024-09-13T04:58:59.961730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"67'264","reported_seq":1221,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:01.924162+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:59:01.924162+0000","last_peered":"2024-09-13T04:59:01.924162+0000","last_clean":"2024-09-13T04:59:01.924162+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:59:01.924162+0000","last_undegraded":"2024-09-13T04:59:01.924162+0000","last_fullsized":"2024-09-13T04:59:01.924162+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":18370745,"num_objects":41,"num_object_clones":5,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":2234,"num_read_kb":9815,"num_write":293,"num_write_kb":4762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:59:00.004990+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:59:00.004990+0000","last_peered":"2024-09-13T04:59:00.004990+0000","last_clean":"2024-09-13T04:59:00.004990+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:59:00.004990+0000","last_undegraded":"2024-09-13T04:59:00.004990+0000","last_fullsized":"2024-09-13T04:59:00.004990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"67'403","reported_seq":605,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.500321+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:59:02.500321+0000","last_peered":"2024-09-13T04:59:02.500321+0000","last_clean":"2024-09-13T04:59:02.500321+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:59:02.500321+0000","last_undegraded":"2024-09-13T04:59:02.500321+0000","last_fullsized":"2024-09-13T04:59:02.500321+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":14898156,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":1477,"num_read_kb":6342,"num_write":587,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":60,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:59:00.008858+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:59:00.008858+0000","last_peered":"2024-09-13T04:59:00.008858+0000","last_clean":"2024-09-13T04:59:00.008858+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:59:00.008858+0000","last_undegraded":"2024-09-13T04:59:00.008858+0000","last_fullsized":"2024-09-13T04:59:00.008858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"67'328","reported_seq":1010,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.896058+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:59:02.896058+0000","last_peered":"2024-09-13T04:59:02.896058+0000","last_clean":"2024-09-13T04:59:02.896058+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:59:02.896058+0000","last_undegraded":"2024-09-13T04:59:02.896058+0000","last_fullsized":"2024-09-13T04:59:02.896058+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":13638194,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":2138,"num_read_kb":7724,"num_write":397,"num_write_kb":3730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"67'330","reported_seq":2679,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.703815+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:59:02.703815+0000","last_peered":"2024-09-13T04:59:02.703815+0000","last_clean":"2024-09-13T04:59:02.703815+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:59:02.703815+0000","last_undegraded":"2024-09-13T04:59:02.703815+0000","last_fullsized":"2024-09-13T04:59:02.703815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":41859594,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":33,"num_read":3337,"num_read_kb":17380,"num_write":381,"num_write_kb":13717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:59:00.004612+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:59:00.004612+0000","last_peered":"2024-09-13T04:59:00.004612+0000","last_clean":"2024-09-13T04:59:00.004612+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:59:00.004612+0000","last_undegraded":"2024-09-13T04:59:00.004612+0000","last_fullsized":"2024-09-13T04:59:00.004612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"67'411","reported_seq":724,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:01.888481+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:59:01.888481+0000","last_peered":"2024-09-13T04:59:01.888481+0000","last_clean":"2024-09-13T04:59:01.888481+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:59:01.888481+0000","last_undegraded":"2024-09-13T04:59:01.888481+0000","last_fullsized":"2024-09-13T04:59:01.888481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":13380216,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":27,"num_read":1596,"num_read_kb":7222,"num_write":557,"num_write_kb":4707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:58:59.967528+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:58:59.967528+0000","last_peered":"2024-09-13T04:58:59.967528+0000","last_clean":"2024-09-13T04:58:59.967528+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:58:59.967528+0000","last_undegraded":"2024-09-13T04:58:59.967528+0000","last_fullsized":"2024-09-13T04:58:59.967528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"67'191","reported_seq":4030,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.708445+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:59:02.708445+0000","last_peered":"2024-09-13T04:59:02.708445+0000","last_clean":"2024-09-13T04:59:02.708445+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:59:02.708445+0000","last_undegraded":"2024-09-13T04:59:02.708445+0000","last_fullsized":"2024-09-13T04:59:02.708445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":39283580,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":4183,"num_read_kb":21293,"num_write":180,"num_write_kb":6183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:59:00.004388+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:59:00.004388+0000","last_peered":"2024-09-13T04:59:00.004388+0000","last_clean":"2024-09-13T04:59:00.004388+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:59:00.004388+0000","last_undegraded":"2024-09-13T04:59:00.004388+0000","last_fullsized":"2024-09-13T04:59:00.004388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"67'496","reported_seq":1582,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.916786+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:59:02.916786+0000","last_peered":"2024-09-13T04:59:02.916786+0000","last_clean":"2024-09-13T04:59:02.916786+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:59:02.916786+0000","last_undegraded":"2024-09-13T04:59:02.916786+0000","last_fullsized":"2024-09-13T04:59:02.916786+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":17705286,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":3611,"num_read_kb":17325,"num_write":710,"num_write_kb":6989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":61,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:59:00.004576+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:59:00.004576+0000","last_peered":"2024-09-13T04:59:00.004576+0000","last_clean":"2024-09-13T04:59:00.004576+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:59:00.004576+0000","last_undegraded":"2024-09-13T04:59:00.004576+0000","last_fullsized":"2024-09-13T04:59:00.004576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":207,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T04:58:59.960812+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:58:59.960812+0000","last_peered":"2024-09-13T04:58:59.960812+0000","last_clean":"2024-09-13T04:58:59.960812+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:58:59.960812+0000","last_undegraded":"2024-09-13T04:58:59.960812+0000","last_fullsized":"2024-09-13T04:58:59.960812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"67'185","reported_seq":982,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.792339+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:59:02.792339+0000","last_peered":"2024-09-13T04:59:02.792339+0000","last_clean":"2024-09-13T04:59:02.792339+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:59:02.792339+0000","last_undegraded":"2024-09-13T04:59:02.792339+0000","last_fullsized":"2024-09-13T04:59:02.792339+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":8480050,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":1609,"num_read_kb":3918,"num_write":158,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"67'423","reported_seq":5185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.712595+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:59:02.712595+0000","last_peered":"2024-09-13T04:59:02.712595+0000","last_clean":"2024-09-13T04:59:02.712595+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:59:02.712595+0000","last_undegraded":"2024-09-13T04:59:02.712595+0000","last_fullsized":"2024-09-13T04:59:02.712595+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":22081440,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":6264,"num_read_kb":20349,"num_write":518,"num_write_kb":9622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"67'229","reported_seq":4606,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.788073+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:59:02.788073+0000","last_peered":"2024-09-13T04:59:02.788073+0000","last_clean":"2024-09-13T04:59:02.788073+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:59:02.788073+0000","last_undegraded":"2024-09-13T04:59:02.788073+0000","last_fullsized":"2024-09-13T04:59:02.788073+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":17005794,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":4584,"num_read_kb":18089,"num_write":239,"num_write_kb":11789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"67'176","reported_seq":607,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.598282+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:59:02.598282+0000","last_peered":"2024-09-13T04:59:02.598282+0000","last_clean":"2024-09-13T04:59:02.598282+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:59:02.598282+0000","last_undegraded":"2024-09-13T04:59:02.598282+0000","last_fullsized":"2024-09-13T04:59:02.598282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":30764537,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":451,"num_read_kb":14561,"num_write":172,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"67'423","reported_seq":925,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.626556+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:59:02.626556+0000","last_peered":"2024-09-13T04:59:02.626556+0000","last_clean":"2024-09-13T04:59:02.626556+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:59:02.626556+0000","last_undegraded":"2024-09-13T04:59:02.626556+0000","last_fullsized":"2024-09-13T04:59:02.626556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":14188144,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":31,"num_read":1919,"num_read_kb":7033,"num_write":634,"num_write_kb":4450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"67'258","reported_seq":3451,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.942289+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:59:02.942289+0000","last_peered":"2024-09-13T04:59:02.942289+0000","last_clean":"2024-09-13T04:59:02.942289+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:59:02.942289+0000","last_undegraded":"2024-09-13T04:59:02.942289+0000","last_fullsized":"2024-09-13T04:59:02.942289+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":31010038,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":4414,"num_read_kb":30846,"num_write":428,"num_write_kb":10836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"67'170","reported_seq":618,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.227757+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:59:02.227757+0000","last_peered":"2024-09-13T04:59:02.227757+0000","last_clean":"2024-09-13T04:59:02.227757+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:59:02.227757+0000","last_undegraded":"2024-09-13T04:59:02.227757+0000","last_fullsized":"2024-09-13T04:59:02.227757+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":18062207,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":21,"num_read":758,"num_read_kb":6510,"num_write":154,"num_write_kb":2984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"67'455","reported_seq":5428,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.214432+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:59:02.214432+0000","last_peered":"2024-09-13T04:59:02.214432+0000","last_clean":"2024-09-13T04:59:02.214432+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:59:02.214432+0000","last_undegraded":"2024-09-13T04:59:02.214432+0000","last_fullsized":"2024-09-13T04:59:02.214432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":21049498,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":6484,"num_read_kb":20670,"num_write":666,"num_write_kb":11955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"67'257","reported_seq":4384,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.713755+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:59:02.713755+0000","last_peered":"2024-09-13T04:59:02.713755+0000","last_clean":"2024-09-13T04:59:02.713755+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:59:02.713755+0000","last_undegraded":"2024-09-13T04:59:02.713755+0000","last_fullsized":"2024-09-13T04:59:02.713755+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":15251493,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":38,"num_read":4863,"num_read_kb":6660,"num_write":245,"num_write_kb":1271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"67'261","reported_seq":4060,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.891321+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:59:02.891321+0000","last_peered":"2024-09-13T04:59:02.891321+0000","last_clean":"2024-09-13T04:59:02.891321+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:59:02.891321+0000","last_undegraded":"2024-09-13T04:59:02.891321+0000","last_fullsized":"2024-09-13T04:59:02.891321+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":45862298,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":4523,"num_read_kb":27346,"num_write":273,"num_write_kb":15650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"67'207","reported_seq":1965,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.516318+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:59:02.516318+0000","last_peered":"2024-09-13T04:59:02.516318+0000","last_clean":"2024-09-13T04:59:02.516318+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:59:02.516318+0000","last_undegraded":"2024-09-13T04:59:02.516318+0000","last_fullsized":"2024-09-13T04:59:02.516318+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":12960962,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":19,"num_read":1873,"num_read_kb":9756,"num_write":221,"num_write_kb":5566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"67'460","reported_seq":1290,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T04:59:02.792277+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:59:02.792277+0000","last_peered":"2024-09-13T04:59:02.792277+0000","last_clean":"2024-09-13T04:59:02.792277+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:59:02.792277+0000","last_undegraded":"2024-09-13T04:59:02.792277+0000","last_fullsized":"2024-09-13T04:59:02.792277+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":12982401,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":27,"num_read":2817,"num_read_kb":8647,"num_write":637,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":710575457,"num_objects":1590,"num_object_clones":393,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1590,"num_whiteouts":815,"num_read":105331,"num_read_kb":494780,"num_write":11533,"num_write_kb":231597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":42074112,"data_stored":342557986,"data_compressed":10806070,"data_compressed_allocated":32808960,"data_compressed_original":335717346,"omap_allocated":0,"internal_metadata":0},"log_size":9388,"ondisk_log_size":9388,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34120,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337720,"statfs":{"total":96636764160,"available":96601825280,"internally_reserved":0,"allocated":3076096,"data_stored":16569445,"data_compressed":671254,"data_compressed_allocated":1708032,"data_compressed_original":15805053,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.80800000000000005}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.0529999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.236},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.121}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.2969999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":1.45},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.337}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.843},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.6259999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41264,"kb_used_data":6052,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330576,"statfs":{"total":96636764160,"available":96594509824,"internally_reserved":0,"allocated":6197248,"data_stored":45689193,"data_compressed":1297485,"data_compressed_allocated":4096000,"data_compressed_original":44376872,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.86799999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.71899999999999997}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.113}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.01}]}]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":6200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":6348800,"data_stored":47088312,"data_compressed":1682325,"data_compressed_allocated":4644864,"data_compressed_original":46080843,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":11.266,"5min":11.266,"15min":11.266},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.95599999999999996}]},{"osd":7,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.96799999999999997}]}]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34584,"kb_used_data":7568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337256,"statfs":{"total":96636764160,"available":96601350144,"internally_reserved":0,"allocated":7749632,"data_stored":57129627,"data_compressed":1500900,"data_compressed_allocated":5492736,"data_compressed_original":55664443,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.30199999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.33200000000000002}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.375}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.111}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.47799999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.80400000000000005}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.67500000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48824,"kb_used_data":5424,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94323016,"statfs":{"total":96636764160,"available":96586768384,"internally_reserved":0,"allocated":5554176,"data_stored":43236352,"data_compressed":1355219,"data_compressed_allocated":3760128,"data_compressed_original":42150951,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.378}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.23400000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.48299999999999998}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.503},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.54800000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47356,"kb_used_data":8008,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94324484,"statfs":{"total":96636764160,"available":96588271616,"internally_reserved":0,"allocated":8200192,"data_stored":60041008,"data_compressed":1626445,"data_compressed_allocated":6057984,"data_compressed_original":58694162,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.77800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":6156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":6303744,"data_stored":45455827,"data_compressed":1517002,"data_compressed_allocated":4177920,"data_compressed_original":44082012,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":15.755000000000001,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.317}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":15.802,"5min":15.802,"15min":15.802},"last":0.40899999999999997}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1,"5min":1,"15min":1},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":1.794,"15min":1.794},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":15.859999999999999,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":11.474,"5min":11.474,"15min":11.474},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":11.528,"5min":11.528,"15min":11.528},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.897,"5min":1.897,"15min":1.897},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":15.885999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":15.834,"5min":15.834,"15min":15.834},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.72299999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40072,"kb_used_data":4864,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331768,"statfs":{"total":96636764160,"available":96595730432,"internally_reserved":0,"allocated":4980736,"data_stored":31281416,"data_compressed":1151472,"data_compressed_allocated":2977792,"data_compressed_original":30022178,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.633,"5min":4.633,"15min":4.633},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.97799999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.6989999999999998,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.071}]},{"osd":6,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.875}]},{"osd":7,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.3520000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5509120,"data_stored":45018793,"data_compressed":1516512,"data_compressed_allocated":4149248,"data_compressed_original":43983708,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7389184,"data_stored":59136777,"data_compressed":1620346,"data_compressed_allocated":6033408,"data_compressed_original":58145298,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4816896,"data_stored":42928882,"data_compressed":1355303,"data_compressed_allocated":3768320,"data_compressed_original":42167335,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6893568,"data_stored":56590323,"data_compressed":1499934,"data_compressed_allocated":5435392,"data_compressed_original":55484219,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5570560,"data_stored":46743568,"data_compressed":1682325,"data_compressed_allocated":4644864,"data_compressed_original":46080843,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":30674926,"data_compressed":1163142,"data_compressed_allocated":2990080,"data_compressed_original":29772322,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5427200,"data_stored":45321229,"data_compressed":1297496,"data_compressed_allocated":4100096,"data_compressed_original":44352296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2273280,"data_stored":16143488,"data_compressed":671012,"data_compressed_allocated":1687552,"data_compressed_original":15731325,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:59:07.067 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [4, 0, 5, 7] 2024-09-13T04:59:07.067 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.3', '4', '0', '5', '7'] 2024-09-13T04:59:07.067 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.3 4 0 5 7 2024-09-13T04:59:07.067 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:07.068 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:07.068 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:59:07.069 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:07.055+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:07.115 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone25 from 0xdf7a21b (overlap 0x1a0d5f8) to 0xa6bbfc 2024-09-13T04:59:07.161 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:07.159+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:07.236 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:07.236 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:07.236 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:07.236 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:07.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:07.258 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-13T04:59:07.262 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:07.258+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:07.307 INFO:teuthology.orchestra.run.smithi001.stdout:3693 read 0x4c01de2 thru 0x4c0cf59 (0xb178 bytes) 2024-09-13T04:59:07.308 INFO:teuthology.orchestra.run.smithi001.stdout:3694 read 0x2da3fa0 thru 0x2dacc94 (0x8cf5 bytes) 2024-09-13T04:59:07.310 INFO:teuthology.orchestra.run.smithi001.stdout:3695 read 0x11db270 thru 0x11e7286 (0xc017 bytes) 2024-09-13T04:59:07.311 INFO:teuthology.orchestra.run.smithi001.stdout:3696 punch from 0x239a15e to 0x23a8501, (0xe3a3 bytes) 2024-09-13T04:59:07.317 INFO:teuthology.orchestra.run.smithi001.stdout:3697 punch from 0x7a94259 to 0x7a9cd06, (0x8aad bytes) 2024-09-13T04:59:07.317 INFO:teuthology.orchestra.run.smithi001.stdout:3699 punch from 0xbe93a01 to 0xbe9ddb5, (0xa3b4 bytes) 2024-09-13T04:59:07.317 INFO:teuthology.orchestra.run.smithi001.stdout:3701 writesame 0x12b3abb thru 0x12c22e3 data_size 0xb(0xe829 bytes) 2024-09-13T04:59:07.348 INFO:teuthology.orchestra.run.smithi001.stdout:3703 trunc from 0xdf7a21b to 0x34534ea 2024-09-13T04:59:07.363 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:07.359+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:07.385 INFO:teuthology.orchestra.run.smithi001.stdout:3704 write 0x469e7b5 thru 0x46ad778 (0xefc4 bytes) 2024-09-13T04:59:07.387 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:07.388 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:07.395 INFO:teuthology.orchestra.run.smithi001.stdout:3705 read 0x3f6b4af thru 0x3f73c27 (0x8779 bytes) 2024-09-13T04:59:07.395 INFO:teuthology.orchestra.run.smithi001.stdout:3707 read 0x426a958 thru 0x427563b (0xace4 bytes) 2024-09-13T04:59:07.405 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:07.412 INFO:teuthology.orchestra.run.smithi001.stdout:3709 writesame 0xb8fa244 thru 0xb9016e5 data_size 0x1(0x74a2 bytes) 2024-09-13T04:59:07.412 INFO:teuthology.orchestra.run.smithi001.stdout:3710 write 0xaab7b45 thru 0xaac6e58 (0xf314 bytes) 2024-09-13T04:59:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:3711 writesame 0xbd08b4e thru 0xbd17676 data_size 0x3(0xeb29 bytes) 2024-09-13T04:59:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:07.431 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:07.442 INFO:teuthology.orchestra.run.smithi001.stdout:3718 write 0xe593560 thru 0xe59a7b4 (0x7255 bytes) 2024-09-13T04:59:07.449 INFO:teuthology.orchestra.run.smithi001.stdout:3719 read 0x2018696 thru 0x201fdb6 (0x7721 bytes) 2024-09-13T04:59:07.450 INFO:teuthology.orchestra.run.smithi001.stdout:3720 write 0x736b160 thru 0x736c655 (0x14f6 bytes) 2024-09-13T04:59:07.456 INFO:teuthology.orchestra.run.smithi001.stdout:3722 write 0x77b05cf thru 0x77b86aa (0x80dc bytes) 2024-09-13T04:59:07.464 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:07.459+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:07.465 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:07.465 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:07.465 INFO:teuthology.orchestra.run.smithi001.stdout:3727 write 0xd59e504 thru 0xd59fa9b (0x1598 bytes) 2024-09-13T04:59:07.471 INFO:teuthology.orchestra.run.smithi001.stdout:3728 write 0x41380c0 thru 0x41428f8 (0xa839 bytes) 2024-09-13T04:59:07.482 INFO:teuthology.orchestra.run.smithi001.stdout:3732 write 0x1c70641 thru 0x1c779b2 (0x7372 bytes) 2024-09-13T04:59:07.493 INFO:teuthology.orchestra.run.smithi001.stdout:3735 writesame 0xc20e508 thru 0xc21a162 data_size 0x1(0xbc5b bytes) 2024-09-13T04:59:07.532 INFO:teuthology.orchestra.run.smithi001.stdout:3736 punch from 0x4474a8f to 0x4475ac3, (0x1034 bytes) 2024-09-13T04:59:07.537 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:59:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.368594791000007, 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:59:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:59:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:59:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:59:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:59:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:59:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:59:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:59:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:59:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:59:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:59:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:59:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:59:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.366154131999998, 2024-09-13T04:59:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:59:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.076921837999997, 2024-09-13T04:59:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:59:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:59:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:07.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:59:07.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:07.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:59:07.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:59:07.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:59:07.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:59:07.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:59:07.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:59:07.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:59:07.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:59:07.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:59:07.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:59:07.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:59:07.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:59:07.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:59:07.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:07.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:59:07.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:59:07.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:07.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:59:07.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:59:07.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:07.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:59:07.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:59:07.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:59:07.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:59:07.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.519322191000001, 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:07.563 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:07.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:59:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:59:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.504811138999997, 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:07.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:59:07.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.574 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70501 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 901120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265763145000001, 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048712395999999998, 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70501 2024-09-13T04:59:07.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266809+0000", 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266812+0000", 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3970000000000001e-06 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266816+0000", 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.729e-06 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266820+0000", 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0690000000000003e-06 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315355+0000", 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048534985000000003 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315417+0000", 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1765000000000002e-05 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315522+0000", 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105142 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70502 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 905216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265727409, 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048974827999999998, 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70502 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266845+0000", 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266851+0000", 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266857+0000", 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7180000000000001e-06 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266861+0000", 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6569999999999996e-06 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315576+0000", 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048714396 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315687+0000", 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111118 2024-09-13T04:59:07.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315821+0000", 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013375400000000001 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70503 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 860160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265674274999999, 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049096338000000003, 2024-09-13T04:59:07.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70503 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266898+0000", 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266901+0000", 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266906+0000", 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4600000000000002e-06 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266911+0000", 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7570000000000004e-06 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315859+0000", 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048947955000000001 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315920+0000", 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0906000000000003e-05 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315995+0000", 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5185000000000006e-05 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70504 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 864256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265658648999999, 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049340255, 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70504 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266917+0000", 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7180000000000001e-06 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266922+0000", 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.028e-06 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266926+0000", 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4750000000000004e-06 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316035+0000", 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049109070999999997 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316136+0000", 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010061 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316255+0000", 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118884 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70505 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 868352~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265643313999998, 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049486095000000001, 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70505 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266929+0000", 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266932+0000", 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.306e-06 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266936+0000", 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8230000000000002e-06 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266941+0000", 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.279e-06 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316291+0000", 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049349968000000001 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316344+0000", 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3659999999999998e-05 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316416+0000", 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1606999999999994e-05 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70506 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 856064~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265628831000001, 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049736427, 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70506 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266946+0000", 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3240000000000001e-06 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266951+0000", 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7489999999999998e-06 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266956+0000", 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9180000000000002e-06 2024-09-13T04:59:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316460+0000", 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049503783000000003 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316555+0000", 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316681+0000", 2024-09-13T04:59:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125513 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70507 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 872448~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265598695000001, 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049868537999999997, 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70507 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266977+0000", 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8459999999999999e-06 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266982+0000", 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3179999999999997e-06 2024-09-13T04:59:07.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266986+0000", 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4320000000000001e-06 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316715+0000", 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049728886 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316772+0000", 2024-09-13T04:59:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6805000000000002e-05 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316843+0000", 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0928999999999998e-05 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70508 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 909312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265571502, 2024-09-13T04:59:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050098392999999998, 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70508 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267001+0000", 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267004+0000", 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2589999999999999e-06 2024-09-13T04:59:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267009+0000", 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4410000000000001e-06 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267014+0000", 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6319999999999999e-06 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316884+0000", 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049869956 2024-09-13T04:59:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316982+0000", 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8472000000000002e-05 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317100+0000", 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117977 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70509 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 876544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265556091000001, 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050242336999999998, 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70509 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267019+0000", 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1969999999999999e-06 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267024+0000", 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267028+0000", 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2169999999999996e-06 2024-09-13T04:59:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317133+0000", 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050105246999999999 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317190+0000", 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6623000000000003e-05 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317259+0000", 2024-09-13T04:59:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9599999999999998e-05 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70510 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 913408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265541537000001, 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050487833000000003, 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70510 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267031+0000", 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267033+0000", 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5660000000000001e-06 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267038+0000", 2024-09-13T04:59:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7650000000000001e-06 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267042+0000", 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.233e-06 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317301+0000", 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050258540999999997 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317403+0000", 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101727 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317519+0000", 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116685 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70511 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 884736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265527377000002, 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050628583999999997, 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70511 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267045+0000", 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267048+0000", 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1430000000000001e-06 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267054+0000", 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7950000000000002e-06 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267059+0000", 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.318e-06 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317553+0000", 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050494202000000002 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317605+0000", 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2039000000000001e-05 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317674+0000", 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999996e-05 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70512 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 880640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265475565999999, 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050833717, 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70512 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267097+0000", 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267100+0000", 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7630000000000001e-06 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267105+0000", 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2850000000000002e-06 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267109+0000", 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5070000000000002e-06 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317716+0000", 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050606774 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317810+0000", 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3851000000000007e-05 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317931+0000", 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121177 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70513 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 892928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265459863, 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050973768000000003, 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70513 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267115+0000", 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2139999999999999e-06 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267122+0000", 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9639999999999997e-06 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267127+0000", 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4789999999999998e-06 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317967+0000", 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050840242000000001 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318019+0000", 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.189e-05 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318087+0000", 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8312999999999995e-05 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70514 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265442790000002, 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051225729999999997, 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70514 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267136+0000", 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1399999999999997e-06 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267141+0000", 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0690000000000001e-06 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267145+0000", 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3830000000000002e-06 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318126+0000", 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050980481000000001 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318237+0000", 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110851 2024-09-13T04:59:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318356+0000", 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119499 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70515 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 897024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.265390754999999, 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051324634000000001, 2024-09-13T04:59:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70515 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267184+0000", 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6529999999999998e-06 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267190+0000", 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267194+0000", 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4859999999999999e-06 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318381+0000", 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051186785999999998 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318436+0000", 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5180999999999999e-05 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318507+0000", 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0753999999999994e-05 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:07.641 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:07.642 INFO:teuthology.orchestra.run.smithi001.stdout:3737 trunc from 0xe59a7b5 to 0x671d815 2024-09-13T04:59:07.643 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:07.644 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:07.644 INFO:teuthology.orchestra.run.smithi001.stdout:3740 punch from 0x527d50b to 0x527faec, (0x25e1 bytes) 2024-09-13T04:59:07.645 INFO:teuthology.orchestra.run.smithi001.stdout:3741 write 0xdab0358 thru 0xdab702f (0x6cd8 bytes) 2024-09-13T04:59:07.645 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:07.645 INFO:teuthology.orchestra.run.smithi001.stdout:3744 punch from 0x2e1b3 to 0x354fe, (0x734b bytes) 2024-09-13T04:59:07.645 INFO:teuthology.orchestra.run.smithi001.stdout:3745 writesame 0xe6d8944 thru 0xe6e3e2a data_size 0x2b(0xb4e7 bytes) 2024-09-13T04:59:07.645 INFO:teuthology.orchestra.run.smithi001.stdout:3748 write 0x65750c1 thru 0x6585053 (0xff93 bytes) 2024-09-13T04:59:07.645 INFO:teuthology.orchestra.run.smithi001.stdout:3749 write 0xcb28576 thru 0xcb38453 (0xfede bytes) 2024-09-13T04:59:07.645 INFO:teuthology.orchestra.run.smithi001.stdout:3750 write 0xad66bbc thru 0xad6b1f3 (0x4638 bytes) 2024-09-13T04:59:07.645 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:07.639+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:07.646 INFO:teuthology.orchestra.run.smithi001.stdout:3754 writesame 0x6a4ca71 thru 0x6a5af8b data_size 0x1(0xe51b bytes) 2024-09-13T04:59:07.734 INFO:teuthology.orchestra.run.smithi001.stdout:3755 write 0xa646019 thru 0xa647c6e (0x1c56 bytes) 2024-09-13T04:59:07.744 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:07.739+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:07.746 INFO:teuthology.orchestra.run.smithi001.stdout:3757 write 0x5c40700 thru 0x5c4ba76 (0xb377 bytes) 2024-09-13T04:59:07.758 INFO:teuthology.orchestra.run.smithi001.stdout:3758 write 0x962e6 thru 0x96777 (0x492 bytes) 2024-09-13T04:59:07.776 INFO:teuthology.orchestra.run.smithi001.stdout:3759 write 0x14cb1b8 thru 0x14d7118 (0xbf61 bytes) 2024-09-13T04:59:07.776 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:07.777 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:07.777 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:07.777 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:07.793 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:07.798 INFO:teuthology.orchestra.run.smithi001.stdout:3760 read 0x7aac1ef thru 0x7ab901c (0xce2e bytes) 2024-09-13T04:59:07.827 INFO:teuthology.orchestra.run.smithi001.stdout:3761 trunc from 0xe6e3e2b to 0x3a62eec 2024-09-13T04:59:07.845 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:07.842+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:07.886 INFO:teuthology.orchestra.run.smithi001.stdout:3762 trunc from 0x3a62eec to 0xcfcd2bb 2024-09-13T04:59:07.898 INFO:teuthology.orchestra.run.smithi001.stdout:3764 read 0x8d35f10 thru 0x8d3babf (0x5bb0 bytes) 2024-09-13T04:59:07.898 INFO:teuthology.orchestra.run.smithi001.stdout:3766 write 0xa72392d thru 0xa7277e0 (0x3eb4 bytes) 2024-09-13T04:59:07.914 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:07.914 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:07.914 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:07.914 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:07.914 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:07.928 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:07.931 INFO:teuthology.orchestra.run.smithi001.stdout:3768 trunc from 0xcfcd2bb to 0x216b9ef 2024-09-13T04:59:07.945 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:07.943+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:08.006 INFO:teuthology.orchestra.run.smithi001.stdout:3770 write 0x5e4dea8 thru 0x5e4f3fb (0x1554 bytes) 2024-09-13T04:59:08.012 INFO:teuthology.orchestra.run.smithi001.stdout:3771 punch from 0x2fd36e to 0x30d2eb, (0xff7d bytes) 2024-09-13T04:59:08.012 INFO:teuthology.orchestra.run.smithi001.stdout:3773 read 0x31a6588 thru 0x31a9c52 (0x36cb bytes) 2024-09-13T04:59:08.012 INFO:teuthology.orchestra.run.smithi001.stdout:3774 writesame 0x3ef70a0 thru 0x3f03ccf data_size 0x18(0xcc30 bytes) 2024-09-13T04:59:08.017 INFO:teuthology.orchestra.run.smithi001.stderr:set 3.3 pg_upmap_items mapping to [4->0,5->7] 2024-09-13T04:59:08.033 INFO:teuthology.orchestra.run.smithi001.stdout:3775 write 0x61c541b thru 0x61cd786 (0x836c bytes) 2024-09-13T04:59:08.046 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:08.042+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:08.057 INFO:teuthology.orchestra.run.smithi001.stdout:3776 write 0xb15f980 thru 0xb16c581 (0xcc02 bytes) 2024-09-13T04:59:08.066 INFO:teuthology.orchestra.run.smithi001.stdout:3779 writesame 0x14af0a8 thru 0x14ba684 data_size 0x3(0xb5dd bytes) 2024-09-13T04:59:08.092 INFO:teuthology.orchestra.run.smithi001.stdout:3780 writesame 0x55d72d9 thru 0x55dc88c data_size 0xa(0x55b4 bytes) 2024-09-13T04:59:08.102 INFO:teuthology.orchestra.run.smithi001.stdout:3781 read 0xa6b87ca thru 0xa6b920a (0xa41 bytes) 2024-09-13T04:59:08.102 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:08.102 INFO:teuthology.orchestra.run.smithi001.stdout:3784 trunc from 0xb16c582 to 0xaace002 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.244339444, 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.105 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T04:59:08.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.244312533, 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T04:59:08.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.240566897000001, 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T04:59:08.114 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.118 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.237023572, 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T04:59:08.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.236983944, 2024-09-13T04:59:08.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T04:59:08.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T04:59:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T04:59:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T04:59:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.236960284, 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.132 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.236872735, 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T04:59:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.236820078999999, 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T04:59:08.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.232894188, 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:08.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T04:59:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T04:59:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.232868563, 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T04:59:08.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.232835168999999, 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:08.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.157 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.22804524, 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T04:59:08.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.224650611, 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T04:59:08.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.166 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.224577934999999, 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.167 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T04:59:08.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.170 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.220835574000001, 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:08.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.175 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.220662582999999, 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:08.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T04:59:08.179 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.216819984000001, 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.180 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T04:59:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T04:59:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T04:59:08.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T04:59:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T04:59:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T04:59:08.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.176972006, 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.186 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.187 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T04:59:08.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T04:59:08.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.172642784000001, 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T04:59:08.190 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T04:59:08.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 14.164913783999999, 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T04:59:08.198 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.199 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.199 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.199 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:08.201 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:08.201 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:08.198+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:08.202 INFO:teuthology.orchestra.run.smithi001.stdout:3786 trunc from 0xaace002 to 0xe5925f8 2024-09-13T04:59:08.202 INFO:teuthology.orchestra.run.smithi001.stdout:3788 write 0xb065502 thru 0xb072f23 (0xda22 bytes) 2024-09-13T04:59:08.202 INFO:teuthology.orchestra.run.smithi001.stdout:3789 read 0x76aa9ac thru 0x76b92a4 (0xe8f9 bytes) 2024-09-13T04:59:08.202 INFO:teuthology.orchestra.run.smithi001.stdout:3791 read 0xacf48f7 thru 0xacfec3b (0xa345 bytes) 2024-09-13T04:59:08.202 INFO:teuthology.orchestra.run.smithi001.stdout:3792 read 0x7a74329 thru 0x7a7433a (0x12 bytes) 2024-09-13T04:59:08.202 INFO:teuthology.orchestra.run.smithi001.stdout:3793 write 0x98ab579 thru 0x98ace32 (0x18ba bytes) 2024-09-13T04:59:08.202 INFO:teuthology.orchestra.run.smithi001.stdout:3795 writesame 0x803b769 thru 0x8048241 data_size 0x1(0xcad9 bytes) 2024-09-13T04:59:08.204 INFO:teuthology.orchestra.run.smithi001.stdout:3796 punch from 0xc9d4fa3 to 0xc9d773d, (0x279a bytes) 2024-09-13T04:59:08.205 INFO:teuthology.orchestra.run.smithi001.stdout:3797 writesame 0xbdcd2c2 thru 0xbddbcf5 data_size 0x1a(0xea34 bytes) 2024-09-13T04:59:08.217 INFO:teuthology.orchestra.run.smithi001.stdout:3798 writesame 0x778b36b thru 0x779980c data_size 0x1(0xe4a2 bytes) 2024-09-13T04:59:08.301 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:08.299+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:08.309 INFO:teuthology.orchestra.run.smithi001.stdout:3799 punch from 0x197f8db to 0x198f0f9, (0xf81e bytes) 2024-09-13T04:59:08.310 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:08.310 INFO:teuthology.orchestra.run.smithi001.stdout:3801 write 0xd7b7fe2 thru 0xd7c40fe (0xc11d bytes) 2024-09-13T04:59:08.341 INFO:teuthology.orchestra.run.smithi001.stdout:3804 trunc from 0xe5925f8 to 0x864d606 2024-09-13T04:59:08.377 INFO:teuthology.orchestra.run.smithi001.stdout:3805 punch from 0x5fd78b4 to 0x5fe5610, (0xdd5c bytes) 2024-09-13T04:59:08.378 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:08.379 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:08.379 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:08.379 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:08.394 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:08.396 INFO:teuthology.orchestra.run.smithi001.stdout:3806 write 0xa45af4f thru 0xa4651d4 (0xa286 bytes) 2024-09-13T04:59:08.402 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:08.399+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:08.403 INFO:teuthology.orchestra.run.smithi001.stdout:3807 read 0x70d4686 thru 0x70dbeb9 (0x7834 bytes) 2024-09-13T04:59:08.403 INFO:teuthology.orchestra.run.smithi001.stdout:3808 punch from 0x17154a3 to 0x1721f99, (0xcaf6 bytes) 2024-09-13T04:59:08.403 INFO:teuthology.orchestra.run.smithi001.stdout:3810 read 0x87d56ed thru 0x87e51da (0xfaee bytes) 2024-09-13T04:59:08.428 INFO:teuthology.orchestra.run.smithi001.stdout:3811 trunc from 0xa4651d5 to 0x173ca4f 2024-09-13T04:59:08.469 INFO:teuthology.orchestra.run.smithi001.stdout:3813 punch from 0x6928c6 to 0x696f20, (0x465a bytes) 2024-09-13T04:59:08.493 INFO:teuthology.orchestra.run.smithi001.stdout:3816 writesame 0x52d9216 thru 0x52e8b21 data_size 0x7e(0xf90c bytes) 2024-09-13T04:59:08.502 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:08.498+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:08.522 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:08.522 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:08.522 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:08.522 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:08.523 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:08.530 INFO:teuthology.orchestra.run.smithi001.stdout:3817 write 0xe427317 thru 0xe429cce (0x29b8 bytes) 2024-09-13T04:59:08.537 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:08.538 INFO:teuthology.orchestra.run.smithi001.stdout:3819 write 0xc768726 thru 0xc775120 (0xc9fb bytes) 2024-09-13T04:59:08.550 INFO:teuthology.orchestra.run.smithi001.stdout:3821 read 0xb344058 thru 0xb3472ad (0x3256 bytes) 2024-09-13T04:59:08.551 INFO:teuthology.orchestra.run.smithi001.stdout:3822 punch from 0x8bae1ec to 0x8bbd26d, (0xf081 bytes) 2024-09-13T04:59:08.551 INFO:teuthology.orchestra.run.smithi001.stdout:3823 write 0x859f645 thru 0x85ac145 (0xcb01 bytes) 2024-09-13T04:59:08.566 INFO:teuthology.orchestra.run.smithi001.stdout:3824 writesame 0xaba807 thru 0xac892d data_size 0x1(0xe127 bytes) 2024-09-13T04:59:08.603 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:08.599+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:08.662 INFO:teuthology.orchestra.run.smithi001.stdout:3827 read 0xc735289 thru 0xc73ab1f (0x5897 bytes) 2024-09-13T04:59:08.664 INFO:teuthology.orchestra.run.smithi001.stdout:3828 trunc from 0xe429ccf to 0xdd1b2ac 2024-09-13T04:59:08.692 INFO:teuthology.orchestra.run.smithi001.stdout:3829 write 0xba5ded9 thru 0xba609af (0x2ad7 bytes) 2024-09-13T04:59:08.702 INFO:teuthology.orchestra.run.smithi001.stdout:3830 punch from 0xc465301 to 0xc46631a, (0x1019 bytes) 2024-09-13T04:59:08.702 INFO:teuthology.orchestra.run.smithi001.stdout:3832 read 0x7028f20 thru 0x702fd7f (0x6e60 bytes) 2024-09-13T04:59:08.702 INFO:teuthology.orchestra.run.smithi001.stdout:3833 write 0x55b75d7 thru 0x55b7975 (0x39f bytes) 2024-09-13T04:59:08.704 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:08.702+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:08.715 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.679153773000003, 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:59:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:59:08.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.679108135999996, 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.721 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:08.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:59:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:59:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:59:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:59:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:59:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:59:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:59:08.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:59:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:59:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.727 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.679060051999997, 2024-09-13T04:59:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:59:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:59:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:59:08.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:59:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:59:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:59:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:59:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:59:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:59:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:59:08.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:59:08.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.678963969999998, 2024-09-13T04:59:08.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:59:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:59:08.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:59:08.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.678912599, 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.741 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:59:08.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.678866544999998, 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.749 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.678816484000002, 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.754 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 52.561401293999999, 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:59:08.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:59:08.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:59:08.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469972910999999, 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:08.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:59:08.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:59:08.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469948565999999, 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:59:08.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469867213000001, 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:08.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469797683000003, 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.776 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:59:08.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469770406999999, 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:08.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:59:08.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:59:08.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.784 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469671962, 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.785 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:59:08.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469602395000003, 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:59:08.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469531920999998, 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.795 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:59:08.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:59:08.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469506236000001, 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.800 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:59:08.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469471048000003, 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:59:08.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:08.809 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469446531999999, 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:59:08.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.469377195, 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:08.814 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:08.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:59:08.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.819 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:08.819 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:08.819 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:08.821 INFO:teuthology.orchestra.run.smithi001.stdout:3835 trunc from 0xdd1b2ac to 0x9d381d7 2024-09-13T04:59:08.821 INFO:teuthology.orchestra.run.smithi001.stdout:3838 write 0x2da6e8c thru 0x2db1dac (0xaf21 bytes) 2024-09-13T04:59:08.821 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:08.821 INFO:teuthology.orchestra.run.smithi001.stdout:3840 read 0x4cda59c thru 0x4ce191a (0x737f bytes) 2024-09-13T04:59:08.821 INFO:teuthology.orchestra.run.smithi001.stdout:3841 write 0x3e995f thru 0x3f0328 (0x69ca bytes) 2024-09-13T04:59:08.821 INFO:teuthology.orchestra.run.smithi001.stdout:3842 writesame 0x5b7fd3c thru 0x5b817f3 data_size 0x6(0x1ab8 bytes) 2024-09-13T04:59:08.821 INFO:teuthology.orchestra.run.smithi001.stdout:3844 writesame 0x69aaaa0 thru 0x69b529b data_size 0x1a(0xa7fc bytes) 2024-09-13T04:59:08.822 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:08.822 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:08.815+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:08.822 INFO:teuthology.orchestra.run.smithi001.stdout:3845 write 0xa45eaf2 thru 0xa46b29b (0xc7aa bytes) 2024-09-13T04:59:08.832 INFO:teuthology.orchestra.run.smithi001.stdout:3846 read 0x384ded0 thru 0x385da42 (0xfb73 bytes) 2024-09-13T04:59:08.833 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:08.849 INFO:teuthology.orchestra.run.smithi001.stdout:3849 trunc from 0xa46b29c to 0x44e5cf3 2024-09-13T04:59:08.885 INFO:teuthology.orchestra.run.smithi001.stdout:3850 trunc from 0x44e5cf3 to 0x7113f01 2024-09-13T04:59:08.898 INFO:teuthology.orchestra.run.smithi001.stdout:3851 read 0xd44847 thru 0xd4bef8 (0x76b2 bytes) 2024-09-13T04:59:08.899 INFO:teuthology.orchestra.run.smithi001.stdout:3852 read 0x6944184 thru 0x6946d02 (0x2b7f bytes) 2024-09-13T04:59:08.899 INFO:teuthology.orchestra.run.smithi001.stdout:3853 read 0x57f4653 thru 0x58033a5 (0xed53 bytes) 2024-09-13T04:59:08.917 INFO:teuthology.orchestra.run.smithi001.stdout:3854 trunc from 0x7113f01 to 0xd47e8d8 2024-09-13T04:59:08.921 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:08.919+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:08.931 INFO:teuthology.orchestra.run.smithi001.stdout:3857 punch from 0x7d5389d to 0x7d5a097, (0x67fa bytes) 2024-09-13T04:59:08.931 INFO:teuthology.orchestra.run.smithi001.stdout:3858 trunc from 0xd47e8d8 to 0xd328d76 2024-09-13T04:59:08.943 INFO:teuthology.orchestra.run.smithi001.stdout:3859 read 0x9339064 thru 0x933be64 (0x2e01 bytes) 2024-09-13T04:59:08.943 INFO:teuthology.orchestra.run.smithi001.stdout:3861 write 0x34d568b thru 0x34dbef5 (0x686b bytes) 2024-09-13T04:59:08.951 INFO:teuthology.orchestra.run.smithi001.stdout:3863 write 0x2bff2fb thru 0x2c04428 (0x512e bytes) 2024-09-13T04:59:08.960 INFO:teuthology.orchestra.run.smithi001.stdout:3864 writesame 0x6d94a34 thru 0x6da0a07 data_size 0x2(0xbfd4 bytes) 2024-09-13T04:59:08.999 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:08.999 INFO:teuthology.orchestra.run.smithi001.stdout:3866 clone 27 order 21 su 8192 sc 2 2024-09-13T04:59:09.001 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:09.002 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:09.002 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:09.017 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:09.022 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:09.019+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:09.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:09.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:59:09.123 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:09.119+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:09.139 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:09.139 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:09.139 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:09.139 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:09.140 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:09.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:09.157 INFO:teuthology.orchestra.run.smithi001.stdout:leaving image image_client.0-clone26 intact 2024-09-13T04:59:09.224 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:09.222+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:09.271 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:09.271 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:09.274 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:09.274 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:09.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:09.274 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473874612000003, 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.275 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:09.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:09.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:09.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:09.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:09.281 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.281 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.281 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.282 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.282 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:09.282 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473840357999997, 2024-09-13T04:59:09.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:09.282 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.282 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.283 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:09.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.284 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:09.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:09.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:09.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:09.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473809457000002, 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:09.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473757575999997, 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.293 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:09.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.297 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473725772999998, 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473691819999999, 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.302 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:09.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:09.306 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473644084, 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.310 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473604522000002, 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:09.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:09.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:09.315 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473556828, 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.316 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:09.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:09.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473493744999999, 2024-09-13T04:59:09.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:09.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.324 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473457310999997, 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.325 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:09.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473423388000001, 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:09.329 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:09.333 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473355175000002, 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.334 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:09.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473321468999998, 2024-09-13T04:59:09.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:09.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.473271277000002, 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:09.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:09.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.472965244000001, 2024-09-13T04:59:09.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:09.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.472916042000001, 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.352 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:09.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.472867927999999, 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:09.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.472816451, 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 45.47278137, 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:09.370 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:09.371 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:09.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:09.372 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:09.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:09.373 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:09.373 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:09.366+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:09.373 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:09.374 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:09.374 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:09.374 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:09.375 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:09.388 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:09.388 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:09.409 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:09.409 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:09.413 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-13T04:59:09.439 INFO:teuthology.orchestra.run.smithi001.stdout:3868 read 0x4e41fab thru 0x4e51a7d (0xfad3 bytes) 2024-09-13T04:59:09.440 INFO:teuthology.orchestra.run.smithi001.stdout:3869 write 0x47c20fa thru 0x47c25f4 (0x4fb bytes) 2024-09-13T04:59:09.450 INFO:teuthology.orchestra.run.smithi001.stdout:3870 punch from 0x67ee400 to 0x67f3dba, (0x59ba bytes) 2024-09-13T04:59:09.450 INFO:teuthology.orchestra.run.smithi001.stdout:3871 read 0xaa4516 thru 0xaac8e9 (0x83d4 bytes) 2024-09-13T04:59:09.451 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:09.468 INFO:teuthology.orchestra.run.smithi001.stdout:3874 trunc from 0xd328d76 to 0x2a9e781 2024-09-13T04:59:09.472 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:09.467+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:09.511 INFO:teuthology.orchestra.run.smithi001.stdout:3875 read 0x2a833db thru 0x2a90e58 (0xda7e bytes) 2024-09-13T04:59:09.512 INFO:teuthology.orchestra.run.smithi001.stdout:3876 read 0x312875 thru 0x314926 (0x20b2 bytes) 2024-09-13T04:59:09.513 INFO:teuthology.orchestra.run.smithi001.stdout:3878 read 0x26f6b34 thru 0x27046a1 (0xdb6e bytes) 2024-09-13T04:59:09.541 INFO:teuthology.orchestra.run.smithi001.stdout:3879 write 0xc71ec1e thru 0xc722af7 (0x3eda bytes) 2024-09-13T04:59:09.550 INFO:teuthology.orchestra.run.smithi001.stdout:3880 write 0x675961e thru 0x6762d69 (0x974c bytes) 2024-09-13T04:59:09.565 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:09.565 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:09.565 INFO:teuthology.orchestra.run.smithi001.stdout:3881 read 0x742fae9 thru 0x743ded6 (0xe3ee bytes) 2024-09-13T04:59:09.566 INFO:teuthology.orchestra.run.smithi001.stdout:3882 punch from 0x2d7459e to 0x2d7dcbb, (0x971d bytes) 2024-09-13T04:59:09.566 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:09.566 INFO:teuthology.orchestra.run.smithi001.stdout:3884 writesame 0xbbc3d61 thru 0xbbcb59b data_size 0x1(0x783b bytes) 2024-09-13T04:59:09.573 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:09.571+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:09.575 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:09.614 INFO:teuthology.orchestra.run.smithi001.stdout:3885 writesame 0x4caf4a1 thru 0x4cb47c9 data_size 0x1(0x5329 bytes) 2024-09-13T04:59:09.672 INFO:teuthology.orchestra.run.smithi001.stdout:3886 trunc from 0xc722af8 to 0x35c3816 2024-09-13T04:59:09.674 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:09.670+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:09.701 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:09.702 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:09.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:09.724 INFO:teuthology.orchestra.run.smithi001.stdout:3888 write 0x830bbd3 thru 0x8317c57 (0xc085 bytes) 2024-09-13T04:59:09.733 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:09.733 INFO:teuthology.orchestra.run.smithi001.stdout:3890 writesame 0x1cccabc thru 0x1cd4461 data_size 0x1(0x79a6 bytes) 2024-09-13T04:59:09.785 INFO:teuthology.orchestra.run.smithi001.stdout:3891 punch from 0x59227e2 to 0x592a4e9, (0x7d07 bytes) 2024-09-13T04:59:09.785 INFO:teuthology.orchestra.run.smithi001.stdout:3893 writesame 0x11b147a thru 0x11bf9f1 data_size 0x1c(0xe578 bytes) 2024-09-13T04:59:09.805 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:09.802+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:09.877 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:09.874+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:09.887 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:09.887 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:09.901 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:09.978 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:09.974+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:09.985 INFO:teuthology.orchestra.run.smithi001.stdout:3894 read 0x8236bd5 thru 0x823a57d (0x39a9 bytes) 2024-09-13T04:59:09.986 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:09.986 INFO:teuthology.orchestra.run.smithi001.stdout:3896 write 0x34071a7 thru 0x3409e0e (0x2c68 bytes) 2024-09-13T04:59:09.994 INFO:teuthology.orchestra.run.smithi001.stdout:3898 read 0x3c98490 thru 0x3ca00b7 (0x7c28 bytes) 2024-09-13T04:59:09.994 INFO:teuthology.orchestra.run.smithi001.stdout:3899 write 0x6207a43 thru 0x6210ef0 (0x94ae bytes) 2024-09-13T04:59:10.004 INFO:teuthology.orchestra.run.smithi001.stdout:3900 read 0xfe4cd0 thru 0xfe75a6 (0x28d7 bytes) 2024-09-13T04:59:10.017 INFO:teuthology.orchestra.run.smithi001.stdout:3906 writesame 0x93f9e4f thru 0x94011a4 data_size 0x1(0x7356 bytes) 2024-09-13T04:59:10.018 INFO:teuthology.orchestra.run.smithi001.stdout:3908 punch from 0x70ab726 to 0x70ac7f8, (0x10d2 bytes) 2024-09-13T04:59:10.018 INFO:teuthology.orchestra.run.smithi001.stdout:3909 writesame 0x16854c8 thru 0x1690f64 data_size 0x2b(0xba9d bytes) 2024-09-13T04:59:10.022 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:59:10.065 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:10.065 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:10.079 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:10.075+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:10.081 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:10.101 INFO:teuthology.orchestra.run.smithi001.stdout:3910 write 0x1581480 thru 0x158ad65 (0x98e6 bytes) 2024-09-13T04:59:10.106 INFO:teuthology.orchestra.run.smithi001.stdout:3911 read 0x58dfa21 thru 0x58e1ec9 (0x24a9 bytes) 2024-09-13T04:59:10.120 INFO:teuthology.orchestra.run.smithi001.stdout:3913 write 0xc3bd2c8 thru 0xc3ccdfe (0xfb37 bytes) 2024-09-13T04:59:10.130 INFO:teuthology.orchestra.run.smithi001.stdout:3914 write 0x6d361a thru 0x6d988c (0x6273 bytes) 2024-09-13T04:59:10.180 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:10.175+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:10.229 INFO:teuthology.orchestra.run.smithi001.stdout:3915 write 0x76b683f thru 0x76b8e6e (0x2630 bytes) 2024-09-13T04:59:10.234 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:10.234 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:10.238 INFO:teuthology.orchestra.run.smithi001.stdout:3917 writesame 0x21872ba thru 0x219028a data_size 0x1(0x8fd1 bytes) 2024-09-13T04:59:10.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:10.282 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:10.278+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:10.283 INFO:teuthology.orchestra.run.smithi001.stdout:3918 trunc from 0xc3ccdff to 0x5133488 2024-09-13T04:59:10.325 INFO:teuthology.orchestra.run.smithi001.stdout:3920 writesame 0x87f0b6d thru 0x87f53a3 data_size 0x13(0x4837 bytes) 2024-09-13T04:59:10.325 INFO:teuthology.orchestra.run.smithi001.stdout:3921 punch from 0x2c9306c to 0x2c9e0e5, (0xb079 bytes) 2024-09-13T04:59:10.326 INFO:teuthology.orchestra.run.smithi001.stdout:3922 flatten 2024-09-13T04:59:10.382 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:10.378+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:10.439 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:10.439 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:10.458 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:10.483 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:10.478+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:10.513 INFO:teuthology.orchestra.run.smithi001.stdout:3923 read 0x7036f74 thru 0x7046c0d (0xfc9a bytes) 2024-09-13T04:59:10.513 INFO:teuthology.orchestra.run.smithi001.stdout:3926 read 0x87d8f1 thru 0x88023f (0x294f bytes) 2024-09-13T04:59:10.514 INFO:teuthology.orchestra.run.smithi001.stdout:3928 read 0x38c7336 thru 0x38c8070 (0xd3b bytes) 2024-09-13T04:59:10.533 INFO:teuthology.orchestra.run.smithi001.stdout:3930 writesame 0xdb9437e thru 0xdb979be data_size 0x13(0x3641 bytes) 2024-09-13T04:59:10.533 INFO:teuthology.orchestra.run.smithi001.stdout:3931 punch from 0xa94d763 to 0xa956541, (0x8dde bytes) 2024-09-13T04:59:10.533 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:10.533 INFO:teuthology.orchestra.run.smithi001.stdout:3936 write 0xc865d49 thru 0xc870c9b (0xaf53 bytes) 2024-09-13T04:59:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:3937 write 0x7f78862 thru 0x7f7963c (0xddb bytes) 2024-09-13T04:59:10.547 INFO:teuthology.orchestra.run.smithi001.stdout:3938 write 0x39e0287 thru 0x39eabb6 (0xa930 bytes) 2024-09-13T04:59:10.557 INFO:teuthology.orchestra.run.smithi001.stdout:3939 read 0x4d2828b thru 0x4d2bd20 (0x3a96 bytes) 2024-09-13T04:59:10.557 INFO:teuthology.orchestra.run.smithi001.stdout:3941 writesame 0x211d942 thru 0x211dc0d data_size 0x2(0x2cc bytes) 2024-09-13T04:59:10.560 INFO:teuthology.orchestra.run.smithi001.stdout:3942 write 0x7d96bd3 thru 0x7d96e50 (0x27e bytes) 2024-09-13T04:59:10.565 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:10.566 INFO:teuthology.orchestra.run.smithi001.stdout:3945 write 0x126d2c thru 0x12f206 (0x84db bytes) 2024-09-13T04:59:10.570 INFO:teuthology.orchestra.run.smithi001.stdout:3946 read 0x369e231 thru 0x36a6940 (0x8710 bytes) 2024-09-13T04:59:10.570 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:10.573 INFO:teuthology.orchestra.run.smithi001.stdout:3948 trunc from 0xdb979bf to 0xc6bed06 2024-09-13T04:59:10.584 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:10.578+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:10.593 INFO:teuthology.orchestra.run.smithi001.stdout:3949 write 0x99e350a thru 0x99f1fda (0xead1 bytes) 2024-09-13T04:59:10.604 INFO:teuthology.orchestra.run.smithi001.stdout:3950 read 0xc54c201 thru 0xc55b5d7 (0xf3d7 bytes) 2024-09-13T04:59:10.604 INFO:teuthology.orchestra.run.smithi001.stdout:3951 read 0x6d1203e thru 0x6d1e9cb (0xc98e bytes) 2024-09-13T04:59:10.618 INFO:teuthology.orchestra.run.smithi001.stdout:3952 write 0xe0aff48 thru 0xe0bfc67 (0xfd20 bytes) 2024-09-13T04:59:10.626 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:10.626 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:10.628 INFO:teuthology.orchestra.run.smithi001.stdout:3953 write 0xb04c69a thru 0xb05720d (0xab74 bytes) 2024-09-13T04:59:10.641 INFO:teuthology.orchestra.run.smithi001.stdout:3954 punch from 0x8315699 to 0x8321764, (0xc0cb bytes) 2024-09-13T04:59:10.641 INFO:teuthology.orchestra.run.smithi001.stdout:3955 read 0xb7efddb thru 0xb7ffc6e (0xfe94 bytes) 2024-09-13T04:59:10.642 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:10.652 INFO:teuthology.orchestra.run.smithi001.stdout:3959 trunc from 0xe0bfc68 to 0x96b987d 2024-09-13T04:59:10.685 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:10.682+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:10.691 INFO:teuthology.orchestra.run.smithi001.stdout:3960 write 0xd633fe2 thru 0xd639f66 (0x5f85 bytes) 2024-09-13T04:59:10.712 INFO:teuthology.orchestra.run.smithi001.stdout:3961 write 0xeb5743e thru 0xeb5d0ed (0x5cb0 bytes) 2024-09-13T04:59:10.721 INFO:teuthology.orchestra.run.smithi001.stdout:3962 write 0x7b74fd3 thru 0x7b75722 (0x750 bytes) 2024-09-13T04:59:10.741 INFO:teuthology.orchestra.run.smithi001.stdout:3963 trunc from 0xeb5d0ee to 0x66c8146 2024-09-13T04:59:10.786 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:10.783+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:10.791 INFO:teuthology.orchestra.run.smithi001.stdout:3964 writesame 0xe06c25d thru 0xe06d073 data_size 0x1(0xe17 bytes) 2024-09-13T04:59:10.791 INFO:teuthology.orchestra.run.smithi001.stdout:3965 writesame 0x65752ba thru 0x65759fd data_size 0x1e(0x744 bytes) 2024-09-13T04:59:10.798 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:10.802 INFO:teuthology.orchestra.run.smithi001.stdout:3967 trunc from 0xe06d074 to 0xba2d688 2024-09-13T04:59:10.813 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:10.813 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:10.828 INFO:teuthology.orchestra.run.smithi001.stdout:3968 write 0xc49e89b thru 0xc4ae83b (0xffa1 bytes) 2024-09-13T04:59:10.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:10.839 INFO:teuthology.orchestra.run.smithi001.stdout:3969 trunc from 0xc4ae83c to 0xbdd120c 2024-09-13T04:59:10.859 INFO:teuthology.orchestra.run.smithi001.stdout:3970 read 0xa6aa318 thru 0xa6b1ff4 (0x7cdd bytes) 2024-09-13T04:59:10.859 INFO:teuthology.orchestra.run.smithi001.stdout:3971 read 0x1c44c33 thru 0x1c4d22e (0x85fc bytes) 2024-09-13T04:59:10.860 INFO:teuthology.orchestra.run.smithi001.stdout:3972 punch from 0x308e382 to 0x3092d3e, (0x49bc bytes) 2024-09-13T04:59:10.860 INFO:teuthology.orchestra.run.smithi001.stdout:3973 punch from 0x5f06ee9 to 0x5f148ed, (0xda04 bytes) 2024-09-13T04:59:10.860 INFO:teuthology.orchestra.run.smithi001.stdout:3974 clone 28 order 19 su 131072 sc 8 2024-09-13T04:59:10.887 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:10.883+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:10.965 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:10.965 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:10.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:11.022 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:11.019+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:11.090 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:11.086+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:11.159 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:11.159 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:11.176 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:11.191 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:11.186+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:11.292 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:11.287+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:11.306 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:11.306 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:11.319 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:11.423 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-13T04:59:11.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:11.433 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:11.427+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:11.525 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:11.522+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:11.571 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:11.571 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:11.591 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:11.598 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:11.616 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:11.625 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:11.619+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:11.633 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:11.636 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:11.650 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:11.672 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:11.693 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:11.693 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:11.714 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:11.714 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:11.724 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:11.724 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:11.725 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:11.723+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:11.735 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:11.735 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:11.741 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:11.826 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:11.822+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:11.849 INFO:teuthology.orchestra.run.smithi001.stdout:3975 read 0x4187fa7 thru 0x4192a2e (0xaa88 bytes) 2024-09-13T04:59:11.851 INFO:teuthology.orchestra.run.smithi001.stdout:3977 read 0x8964c37 thru 0x8971b2a (0xcef4 bytes) 2024-09-13T04:59:11.852 INFO:teuthology.orchestra.run.smithi001.stdout:3978 trunc from 0xbdd120c to 0xc22a0e8 2024-09-13T04:59:11.886 INFO:teuthology.orchestra.run.smithi001.stdout:3979 trunc from 0xc22a0e8 to 0x6720a6 2024-09-13T04:59:11.927 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:11.922+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:11.933 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:11.941 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:11.941 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:11.942 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:11.942 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:11.942 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:11.945 INFO:teuthology.orchestra.run.smithi001.stdout:3983 write 0x2733540 thru 0x2738960 (0x5421 bytes) 2024-09-13T04:59:11.959 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:11.980 INFO:teuthology.orchestra.run.smithi001.stdout:3984 writesame 0xe581ecb thru 0xe584e3c data_size 0x1(0x2f72 bytes) 2024-09-13T04:59:11.990 INFO:teuthology.orchestra.run.smithi001.stdout:3985 trunc from 0xe584e3d to 0x92dd7cf 2024-09-13T04:59:12.000 INFO:teuthology.orchestra.run.smithi001.stdout:3986 read 0x27a6e7c thru 0x27b68e5 (0xfa6a bytes) 2024-09-13T04:59:12.000 INFO:teuthology.orchestra.run.smithi001.stdout:3987 punch from 0x76134a9 to 0x761da71, (0xa5c8 bytes) 2024-09-13T04:59:12.000 INFO:teuthology.orchestra.run.smithi001.stdout:3988 punch from 0x5c790be to 0x5c7a904, (0x1846 bytes) 2024-09-13T04:59:12.010 INFO:teuthology.orchestra.run.smithi001.stdout:3990 trunc from 0x92dd7cf to 0x3d5abd5 2024-09-13T04:59:12.020 INFO:teuthology.orchestra.run.smithi001.stdout:3991 write 0x619e6 thru 0x6851d (0x6b38 bytes) 2024-09-13T04:59:12.027 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:12.022+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:12.033 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.033 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.033 INFO:teuthology.orchestra.run.smithi001.stdout:3994 read 0x2b2a059 thru 0x2b2c36e (0x2316 bytes) 2024-09-13T04:59:12.033 INFO:teuthology.orchestra.run.smithi001.stdout:3996 writesame 0x85bd2c thru 0x86130e data_size 0x15(0x55e3 bytes) 2024-09-13T04:59:12.055 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:59:12.056 INFO:teuthology.orchestra.run.smithi001.stdout:3997 write 0x7d9c916 thru 0x7da1776 (0x4e61 bytes) 2024-09-13T04:59:12.061 INFO:teuthology.orchestra.run.smithi001.stdout:3998 read 0x16c8b75 thru 0x16d6c72 (0xe0fe bytes) 2024-09-13T04:59:12.069 INFO:teuthology.orchestra.run.smithi001.stdout:4002 trunc from 0x7da1777 to 0x37c6a9e 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.927888581999994, 2024-09-13T04:59:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:59:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.925447923, 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:59:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 55.636215628999999, 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:59:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.078615982000002, 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:59:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.064104929999999, 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:59:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:59:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70501 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 901120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.825056935999999, 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048712395999999998, 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70501 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266809+0000", 2024-09-13T04:59:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266812+0000", 2024-09-13T04:59:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3970000000000001e-06 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266816+0000", 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.729e-06 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266820+0000", 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0690000000000003e-06 2024-09-13T04:59:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315355+0000", 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048534985000000003 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315417+0000", 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1765000000000002e-05 2024-09-13T04:59:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315522+0000", 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105142 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70502 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 905216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.825021199999998, 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048974827999999998, 2024-09-13T04:59:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70502 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266845+0000", 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266851+0000", 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266857+0000", 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7180000000000001e-06 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266861+0000", 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6569999999999996e-06 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315576+0000", 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048714396 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315687+0000", 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111118 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315821+0000", 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013375400000000001 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70503 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 860160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824968066, 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049096338000000003, 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70503 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266898+0000", 2024-09-13T04:59:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266901+0000", 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266906+0000", 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4600000000000002e-06 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266911+0000", 2024-09-13T04:59:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7570000000000004e-06 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315859+0000", 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048947955000000001 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315920+0000", 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0906000000000003e-05 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315995+0000", 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5185000000000006e-05 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70504 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 864256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824952440000001, 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049340255, 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70504 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266917+0000", 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7180000000000001e-06 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266922+0000", 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.028e-06 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266926+0000", 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4750000000000004e-06 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316035+0000", 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049109070999999997 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316136+0000", 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010061 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316255+0000", 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118884 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70505 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 868352~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824937105, 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049486095000000001, 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70505 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266929+0000", 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266932+0000", 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.306e-06 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266936+0000", 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8230000000000002e-06 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266941+0000", 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.279e-06 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316291+0000", 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049349968000000001 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316344+0000", 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3659999999999998e-05 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316416+0000", 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1606999999999994e-05 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70506 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 856064~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824922621999999, 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049736427, 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70506 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266946+0000", 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3240000000000001e-06 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266951+0000", 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7489999999999998e-06 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266956+0000", 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9180000000000002e-06 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316460+0000", 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049503783000000003 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316555+0000", 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316681+0000", 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125513 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70507 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 872448~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824892486, 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049868537999999997, 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70507 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266977+0000", 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8459999999999999e-06 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266982+0000", 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3179999999999997e-06 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266986+0000", 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4320000000000001e-06 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316715+0000", 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049728886 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316772+0000", 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6805000000000002e-05 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316843+0000", 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0928999999999998e-05 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70508 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 909312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824865292999998, 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050098392999999998, 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70508 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267001+0000", 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267004+0000", 2024-09-13T04:59:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2589999999999999e-06 2024-09-13T04:59:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267009+0000", 2024-09-13T04:59:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4410000000000001e-06 2024-09-13T04:59:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267014+0000", 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6319999999999999e-06 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316884+0000", 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049869956 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316982+0000", 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8472000000000002e-05 2024-09-13T04:59:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317100+0000", 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117977 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70509 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 876544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824849881999999, 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050242336999999998, 2024-09-13T04:59:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70509 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267019+0000", 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1969999999999999e-06 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267024+0000", 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267028+0000", 2024-09-13T04:59:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2169999999999996e-06 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317133+0000", 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050105246999999999 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317190+0000", 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6623000000000003e-05 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317259+0000", 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9599999999999998e-05 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70510 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 913408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824835327999999, 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050487833000000003, 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70510 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267031+0000", 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267033+0000", 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5660000000000001e-06 2024-09-13T04:59:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267038+0000", 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7650000000000001e-06 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267042+0000", 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.233e-06 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317301+0000", 2024-09-13T04:59:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050258540999999997 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317403+0000", 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101727 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317519+0000", 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116685 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70511 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 884736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824821168, 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050628583999999997, 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70511 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267045+0000", 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267048+0000", 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1430000000000001e-06 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267054+0000", 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7950000000000002e-06 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267059+0000", 2024-09-13T04:59:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.318e-06 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317553+0000", 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050494202000000002 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317605+0000", 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2039000000000001e-05 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317674+0000", 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999996e-05 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70512 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 880640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824769357000001, 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050833717, 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70512 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267097+0000", 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267100+0000", 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7630000000000001e-06 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267105+0000", 2024-09-13T04:59:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2850000000000002e-06 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267109+0000", 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5070000000000002e-06 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317716+0000", 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050606774 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317810+0000", 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3851000000000007e-05 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317931+0000", 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121177 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70513 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 892928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824753653999998, 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050973768000000003, 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70513 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267115+0000", 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2139999999999999e-06 2024-09-13T04:59:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267122+0000", 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9639999999999997e-06 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267127+0000", 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4789999999999998e-06 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317967+0000", 2024-09-13T04:59:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050840242000000001 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318019+0000", 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.189e-05 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318087+0000", 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8312999999999995e-05 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70514 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824736581, 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051225729999999997, 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70514 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267136+0000", 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1399999999999997e-06 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267141+0000", 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0690000000000001e-06 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267145+0000", 2024-09-13T04:59:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3830000000000002e-06 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318126+0000", 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050980481000000001 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318237+0000", 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110851 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318356+0000", 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119499 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70515 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 897024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.824684546, 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051324634000000001, 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70515 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267184+0000", 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6529999999999998e-06 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267190+0000", 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267194+0000", 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4859999999999999e-06 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318381+0000", 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051186785999999998 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318436+0000", 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5180999999999999e-05 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318507+0000", 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0753999999999994e-05 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:12.190 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:12.191 INFO:teuthology.orchestra.run.smithi001.stdout:4003 trunc from 0x37c6a9e to 0x88522d1 2024-09-13T04:59:12.191 INFO:teuthology.orchestra.run.smithi001.stdout:4005 write 0x4c5ad1a thru 0x4c64c1c (0x9f03 bytes) 2024-09-13T04:59:12.192 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:12.193 INFO:teuthology.orchestra.run.smithi001.stdout:4006 trunc from 0x88522d1 to 0xe03375d 2024-09-13T04:59:12.193 INFO:teuthology.orchestra.run.smithi001.stdout:4007 writesame 0xca956b6 thru 0xca9ce19 data_size 0x36(0x7764 bytes) 2024-09-13T04:59:12.193 INFO:teuthology.orchestra.run.smithi001.stdout:4008 write 0x8783fd5 thru 0x87917a4 (0xd7d0 bytes) 2024-09-13T04:59:12.193 INFO:teuthology.orchestra.run.smithi001.stdout:4009 trunc from 0xe03375d to 0xed12569 2024-09-13T04:59:12.193 INFO:teuthology.orchestra.run.smithi001.stdout:4010 write 0x2da5bc7 thru 0x2da813d (0x2577 bytes) 2024-09-13T04:59:12.193 INFO:teuthology.orchestra.run.smithi001.stdout:4011 write 0x6ba9c17 thru 0x6bb653e (0xc928 bytes) 2024-09-13T04:59:12.193 INFO:teuthology.orchestra.run.smithi001.stdout:4012 read 0x76e1fd thru 0x77465c (0x6460 bytes) 2024-09-13T04:59:12.194 INFO:teuthology.orchestra.run.smithi001.stdout:4013 punch from 0x21280e3 to 0x2136c66, (0xeb83 bytes) 2024-09-13T04:59:12.194 INFO:teuthology.orchestra.run.smithi001.stdout:4015 punch from 0x74176f0 to 0x741ad03, (0x3613 bytes) 2024-09-13T04:59:12.194 INFO:teuthology.orchestra.run.smithi001.stdout:4016 read 0xca4cdc thru 0xcaf2f8 (0xa61d bytes) 2024-09-13T04:59:12.194 INFO:teuthology.orchestra.run.smithi001.stdout:4017 read 0x51d0492 thru 0x51dc50c (0xc07b bytes) 2024-09-13T04:59:12.194 INFO:teuthology.orchestra.run.smithi001.stdout:4018 read 0x27b9708 thru 0x27c1500 (0x7df9 bytes) 2024-09-13T04:59:12.194 INFO:teuthology.orchestra.run.smithi001.stdout:4019 read 0x1ff346b thru 0x1ffecf3 (0xb889 bytes) 2024-09-13T04:59:12.195 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:12.190+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:12.202 INFO:teuthology.orchestra.run.smithi001.stdout:4020 trunc from 0xed12569 to 0x1b67274 2024-09-13T04:59:12.236 INFO:teuthology.orchestra.run.smithi001.stdout:4021 writesame 0x5e12693 thru 0x5e2140b data_size 0x1(0xed79 bytes) 2024-09-13T04:59:12.243 INFO:teuthology.orchestra.run.smithi001.stdout:4022 trunc from 0x5e2140c to 0xed0ec7 2024-09-13T04:59:12.269 INFO:teuthology.orchestra.run.smithi001.stdout:4024 write 0x6cd1d03 thru 0x6cdc19a (0xa498 bytes) 2024-09-13T04:59:12.272 INFO:teuthology.orchestra.run.smithi001.stdout:4026 write 0x5f30d5 thru 0x6001ae (0xd0da bytes) 2024-09-13T04:59:12.281 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.281 INFO:teuthology.orchestra.run.smithi001.stdout:4028 read 0x59a0afe thru 0x59a6575 (0x5a78 bytes) 2024-09-13T04:59:12.281 INFO:teuthology.orchestra.run.smithi001.stdout:4029 punch from 0x2eab20d to 0x2eb4031, (0x8e24 bytes) 2024-09-13T04:59:12.282 INFO:teuthology.orchestra.run.smithi001.stdout:4030 punch from 0x4e5991a to 0x4e5ff66, (0x664c bytes) 2024-09-13T04:59:12.282 INFO:teuthology.orchestra.run.smithi001.stdout:4031 read 0x13f4b1a thru 0x13fac6e (0x6155 bytes) 2024-09-13T04:59:12.296 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:12.291+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:12.300 INFO:teuthology.orchestra.run.smithi001.stdout:4032 write 0xc9b16b6 thru 0xc9bd740 (0xc08b bytes) 2024-09-13T04:59:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:4033 read 0xa441cbd thru 0xa4509cd (0xed11 bytes) 2024-09-13T04:59:12.314 INFO:teuthology.orchestra.run.smithi001.stdout:4034 write 0xc9dc8f2 thru 0xc9e31b1 (0x68c0 bytes) 2024-09-13T04:59:12.320 INFO:teuthology.orchestra.run.smithi001.stdout:4035 write 0xb187ee8 thru 0xb190b59 (0x8c72 bytes) 2024-09-13T04:59:12.331 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:12.332 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:12.332 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:12.332 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:12.341 INFO:teuthology.orchestra.run.smithi001.stdout:4036 trunc from 0xc9e31b2 to 0x3de2f5a 2024-09-13T04:59:12.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:12.358 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.375 INFO:teuthology.orchestra.run.smithi001.stdout:4038 writesame 0x814497a thru 0x814f815 data_size 0x1e(0xae9c bytes) 2024-09-13T04:59:12.384 INFO:teuthology.orchestra.run.smithi001.stdout:4039 write 0x838cc4a thru 0x8396618 (0x99cf bytes) 2024-09-13T04:59:12.390 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.390 INFO:teuthology.orchestra.run.smithi001.stdout:4041 punch from 0x2267a5f to 0x2271c67, (0xa208 bytes) 2024-09-13T04:59:12.390 INFO:teuthology.orchestra.run.smithi001.stdout:4042 write 0x32c47a5 thru 0x32d324f (0xeaab bytes) 2024-09-13T04:59:12.396 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:12.391+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:12.401 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.401 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.401 INFO:teuthology.orchestra.run.smithi001.stdout:4045 read 0x75b1c88 thru 0x75b8b60 (0x6ed9 bytes) 2024-09-13T04:59:12.401 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.401 INFO:teuthology.orchestra.run.smithi001.stdout:4048 read 0x61ef0c thru 0x6290ec (0xa1e1 bytes) 2024-09-13T04:59:12.402 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.402 INFO:teuthology.orchestra.run.smithi001.stdout:4051 punch from 0x5080ca7 to 0x508c90d, (0xbc66 bytes) 2024-09-13T04:59:12.402 INFO:teuthology.orchestra.run.smithi001.stdout:4053 writesame 0x55dac59 thru 0x55daeb9 data_size 0x3(0x261 bytes) 2024-09-13T04:59:12.406 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.406 INFO:teuthology.orchestra.run.smithi001.stdout:4057 read 0x63de289 thru 0x63e05d5 (0x234d bytes) 2024-09-13T04:59:12.407 INFO:teuthology.orchestra.run.smithi001.stdout:4058 read 0x62aa5ac thru 0x62b4161 (0x9bb6 bytes) 2024-09-13T04:59:12.424 INFO:teuthology.orchestra.run.smithi001.stdout:4059 write 0xd765c45 thru 0xd766af5 (0xeb1 bytes) 2024-09-13T04:59:12.430 INFO:teuthology.orchestra.run.smithi001.stdout:4061 read 0x5b7454c thru 0x5b8194e (0xd403 bytes) 2024-09-13T04:59:12.430 INFO:teuthology.orchestra.run.smithi001.stdout:4062 read 0xc9ca389 thru 0xc9d60f4 (0xbd6c bytes) 2024-09-13T04:59:12.430 INFO:teuthology.orchestra.run.smithi001.stdout:4063 write 0xba9b0bf thru 0xbaa23f8 (0x733a bytes) 2024-09-13T04:59:12.439 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.440 INFO:teuthology.orchestra.run.smithi001.stdout:4069 read 0x7cfcf18 thru 0x7d0659e (0x9687 bytes) 2024-09-13T04:59:12.440 INFO:teuthology.orchestra.run.smithi001.stdout:4070 write 0x62d2823 thru 0x62dc53b (0x9d19 bytes) 2024-09-13T04:59:12.447 INFO:teuthology.orchestra.run.smithi001.stdout:4071 writesame 0x6a78a32 thru 0x6a81b21 data_size 0x10(0x90f0 bytes) 2024-09-13T04:59:12.456 INFO:teuthology.orchestra.run.smithi001.stdout:4072 write 0x68b1eed thru 0x68bbe99 (0x9fad bytes) 2024-09-13T04:59:12.464 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.465 INFO:teuthology.orchestra.run.smithi001.stdout:4074 read 0x735b62d thru 0x735f725 (0x40f9 bytes) 2024-09-13T04:59:12.465 INFO:teuthology.orchestra.run.smithi001.stdout:4075 writesame 0x77c503e thru 0x77c7141 data_size 0x2(0x2104 bytes) 2024-09-13T04:59:12.468 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:12.468 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:12.469 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:12.469 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:12.472 INFO:teuthology.orchestra.run.smithi001.stdout:4076 read 0x7813156 thru 0x78224a9 (0xf354 bytes) 2024-09-13T04:59:12.473 INFO:teuthology.orchestra.run.smithi001.stdout:4077 punch from 0x3c6f712 to 0x3c6fefa, (0x7e8 bytes) 2024-09-13T04:59:12.473 INFO:teuthology.orchestra.run.smithi001.stdout:4078 punch from 0x875df69 to 0x876008c, (0x2123 bytes) 2024-09-13T04:59:12.473 INFO:teuthology.orchestra.run.smithi001.stdout:4080 write 0x857e948 thru 0x858aeb8 (0xc571 bytes) 2024-09-13T04:59:12.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:12.486 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.486 INFO:teuthology.orchestra.run.smithi001.stdout:4082 read 0x8ebe26d thru 0x8ecd1d1 (0xef65 bytes) 2024-09-13T04:59:12.486 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.486 INFO:teuthology.orchestra.run.smithi001.stdout:4084 read 0x43e10ab thru 0x43eeecf (0xde25 bytes) 2024-09-13T04:59:12.487 INFO:teuthology.orchestra.run.smithi001.stdout:4085 write 0x6900ed2 thru 0x690ecf1 (0xde20 bytes) 2024-09-13T04:59:12.497 INFO:teuthology.orchestra.run.smithi001.stdout:4091 write 0x9fd830c thru 0x9fdb841 (0x3536 bytes) 2024-09-13T04:59:12.498 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:12.495+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:12.503 INFO:teuthology.orchestra.run.smithi001.stdout:4093 read 0x4bc22b3 thru 0x4bc5f89 (0x3cd7 bytes) 2024-09-13T04:59:12.503 INFO:teuthology.orchestra.run.smithi001.stdout:4094 write 0x1c9b960 thru 0x1c9c347 (0x9e8 bytes) 2024-09-13T04:59:12.508 INFO:teuthology.orchestra.run.smithi001.stdout:4095 read 0x7154d25 thru 0x715c75a (0x7a36 bytes) 2024-09-13T04:59:12.512 INFO:teuthology.orchestra.run.smithi001.stdout:4096 trunc from 0xd766af6 to 0xbd5d14c 2024-09-13T04:59:12.530 INFO:teuthology.orchestra.run.smithi001.stdout:4097 writesame 0x6102e4d thru 0x610b632 data_size 0x47(0x87e6 bytes) 2024-09-13T04:59:12.536 INFO:teuthology.orchestra.run.smithi001.stdout:4098 read 0x49f0a6a thru 0x49ffe31 (0xf3c8 bytes) 2024-09-13T04:59:12.536 INFO:teuthology.orchestra.run.smithi001.stdout:4099 write 0xb3c898f thru 0xb3ceab3 (0x6125 bytes) 2024-09-13T04:59:12.542 INFO:teuthology.orchestra.run.smithi001.stdout:4100 punch from 0x333945e to 0x333d703, (0x42a5 bytes) 2024-09-13T04:59:12.542 INFO:teuthology.orchestra.run.smithi001.stdout:4103 punch from 0x78181c2 to 0x781d30b, (0x5149 bytes) 2024-09-13T04:59:12.543 INFO:teuthology.orchestra.run.smithi001.stdout:4104 writesame 0x8c8faf6 thru 0x8c9c74f data_size 0x3(0xcc5a bytes) 2024-09-13T04:59:12.560 INFO:teuthology.orchestra.run.smithi001.stdout:4105 write 0x412d25 thru 0x41772d (0x4a09 bytes) 2024-09-13T04:59:12.565 INFO:teuthology.orchestra.run.smithi001.stdout:4107 writesame 0x98dc55b thru 0x98e8e5d data_size 0x11(0xc903 bytes) 2024-09-13T04:59:12.575 INFO:teuthology.orchestra.run.smithi001.stdout:4108 punch from 0x3bcfb25 to 0x3bd2f36, (0x3411 bytes) 2024-09-13T04:59:12.598 INFO:teuthology.orchestra.run.smithi001.stdout:4109 trunc from 0xbd5d14c to 0x206e76f 2024-09-13T04:59:12.599 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:12.595+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:12.609 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:12.609 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:12.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:12.609 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:12.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.609 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.609 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:12.609 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.748641686999999, 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:12.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T04:59:12.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.748614776, 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.614 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:12.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.744869139999999, 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:12.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T04:59:12.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.741325815, 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T04:59:12.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.627 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.741286187, 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T04:59:12.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.741262527, 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T04:59:12.632 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T04:59:12.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.741174978, 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.637 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T04:59:12.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.741122321999999, 2024-09-13T04:59:12.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T04:59:12.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.645 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.737196431000001, 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.646 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:12.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T04:59:12.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.737170806000002, 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T04:59:12.650 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T04:59:12.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.654 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.737137411999999, 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.655 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T04:59:12.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:12.659 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.732347483000002, 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:12.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.663 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.728952853999999, 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.664 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T04:59:12.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.668 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.728880178000001, 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T04:59:12.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.725137817, 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T04:59:12.673 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T04:59:12.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T04:59:12.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.677 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.724964826000001, 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.678 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.682 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.721122226999999, 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.683 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T04:59:12.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T04:59:12.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.681274249000001, 2024-09-13T04:59:12.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T04:59:12.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.691 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.676945026999999, 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T04:59:12.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T04:59:12.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T04:59:12.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T04:59:12.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.696 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 18.669216027000001, 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T04:59:12.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:12.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:12.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:12.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:12.701 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:12.703 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:12.704 INFO:teuthology.orchestra.run.smithi001.stdout:4110 read 0x11030d7 thru 0x1110162 (0xd08c bytes) 2024-09-13T04:59:12.704 INFO:teuthology.orchestra.run.smithi001.stdout:4111 punch from 0x106c4c6 to 0x1076d4d, (0xa887 bytes) 2024-09-13T04:59:12.704 INFO:teuthology.orchestra.run.smithi001.stdout:4113 write 0x7f798bf thru 0x7f7a289 (0x9cb bytes) 2024-09-13T04:59:12.704 INFO:teuthology.orchestra.run.smithi001.stdout:4115 write 0xc170ed3 thru 0xc1712d7 (0x405 bytes) 2024-09-13T04:59:12.704 INFO:teuthology.orchestra.run.smithi001.stdout:4118 read 0x6553a0b thru 0x655619f (0x2795 bytes) 2024-09-13T04:59:12.704 INFO:teuthology.orchestra.run.smithi001.stdout:4119 read 0x6559ecd thru 0x655de86 (0x3fba bytes) 2024-09-13T04:59:12.704 INFO:teuthology.orchestra.run.smithi001.stdout:4120 punch from 0x19bb8a7 to 0x19c1cfc, (0x6455 bytes) 2024-09-13T04:59:12.705 INFO:teuthology.orchestra.run.smithi001.stdout:4121 write 0xad238f4 thru 0xad330eb (0xf7f8 bytes) 2024-09-13T04:59:12.705 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:12.699+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:12.709 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.709 INFO:teuthology.orchestra.run.smithi001.stdout:4125 writesame 0x85688f3 thru 0x8573ade data_size 0x3b(0xb1ec bytes) 2024-09-13T04:59:12.718 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.732 INFO:teuthology.orchestra.run.smithi001.stdout:4127 write 0xd5e6620 thru 0xd5ec17c (0x5b5d bytes) 2024-09-13T04:59:12.741 INFO:teuthology.orchestra.run.smithi001.stdout:4129 read 0x6569b94 thru 0x6570dd7 (0x7244 bytes) 2024-09-13T04:59:12.742 INFO:teuthology.orchestra.run.smithi001.stdout:4130 read 0x9fae4c1 thru 0x9fb21e6 (0x3d26 bytes) 2024-09-13T04:59:12.742 INFO:teuthology.orchestra.run.smithi001.stdout:4132 write 0x986a81d thru 0x98704b9 (0x5c9d bytes) 2024-09-13T04:59:12.749 INFO:teuthology.orchestra.run.smithi001.stdout:4133 punch from 0x181ab3c to 0x1826131, (0xb5f5 bytes) 2024-09-13T04:59:12.749 INFO:teuthology.orchestra.run.smithi001.stdout:4134 read 0x7894c58 thru 0x7897206 (0x25af bytes) 2024-09-13T04:59:12.749 INFO:teuthology.orchestra.run.smithi001.stdout:4136 write 0x9bee0a6 thru 0x9bf9bc5 (0xbb20 bytes) 2024-09-13T04:59:12.761 INFO:teuthology.orchestra.run.smithi001.stdout:4138 writesame 0x6476046 thru 0x64783d3 data_size 0x29(0x238e bytes) 2024-09-13T04:59:12.769 INFO:teuthology.orchestra.run.smithi001.stdout:4139 write 0x67e7459 thru 0x67e88e2 (0x148a bytes) 2024-09-13T04:59:12.777 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.777 INFO:teuthology.orchestra.run.smithi001.stdout:4143 read 0x63aee9 thru 0x63b26c (0x384 bytes) 2024-09-13T04:59:12.777 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.780 INFO:teuthology.orchestra.run.smithi001.stdout:4145 trunc from 0xd5ec17d to 0xc70992f 2024-09-13T04:59:12.804 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:12.798+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:12.806 INFO:teuthology.orchestra.run.smithi001.stdout:4147 write 0xe8cf18 thru 0xe8db43 (0xc2c bytes) 2024-09-13T04:59:12.813 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.813 INFO:teuthology.orchestra.run.smithi001.stdout:4151 punch from 0xb79da4 to 0xb88950, (0xebac bytes) 2024-09-13T04:59:12.816 INFO:teuthology.orchestra.run.smithi001.stdout:4152 trunc from 0xc70992f to 0xd8434f3 2024-09-13T04:59:12.830 INFO:teuthology.orchestra.run.smithi001.stdout:4153 write 0x7ed9364 thru 0x7ee6045 (0xcce2 bytes) 2024-09-13T04:59:12.836 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:12.836 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:12.836 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:12.844 INFO:teuthology.orchestra.run.smithi001.stdout:4155 write 0x15eccc thru 0x16c0be (0xd3f3 bytes) 2024-09-13T04:59:12.851 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:12.864 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.864 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.864 INFO:teuthology.orchestra.run.smithi001.stdout:4158 read 0x161684d thru 0x161b7e5 (0x4f99 bytes) 2024-09-13T04:59:12.865 INFO:teuthology.orchestra.run.smithi001.stdout:4159 read 0x8a4b952 thru 0x8a52132 (0x67e1 bytes) 2024-09-13T04:59:12.865 INFO:teuthology.orchestra.run.smithi001.stdout:4160 write 0x712b2f5 thru 0x712e648 (0x3354 bytes) 2024-09-13T04:59:12.873 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.873 INFO:teuthology.orchestra.run.smithi001.stdout:4162 writesame 0xb435493 thru 0xb43d700 data_size 0x23(0x826e bytes) 2024-09-13T04:59:12.882 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.882 INFO:teuthology.orchestra.run.smithi001.stdout:4164 read 0x203fbf2 thru 0x2049edc (0xa2eb bytes) 2024-09-13T04:59:12.884 INFO:teuthology.orchestra.run.smithi001.stdout:4165 trunc from 0xd8434f3 to 0xd17435e 2024-09-13T04:59:12.898 INFO:teuthology.orchestra.run.smithi001.stdout:4166 read 0x7f4cc8d thru 0x7f4d094 (0x408 bytes) 2024-09-13T04:59:12.913 INFO:teuthology.orchestra.run.smithi001.stdout:4167 write 0xdcf3a8a thru 0xdcf688f (0x2e06 bytes) 2024-09-13T04:59:12.921 INFO:teuthology.orchestra.run.smithi001.stdout:4168 write 0x75a4818 thru 0x75a9d0f (0x54f8 bytes) 2024-09-13T04:59:12.930 INFO:teuthology.orchestra.run.smithi001.stdout:4170 read 0x12b1e4c thru 0x12c1c99 (0xfe4e bytes) 2024-09-13T04:59:12.930 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.941 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:12.938+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:12.944 INFO:teuthology.orchestra.run.smithi001.stdout:4172 writesame 0xe7f5995 thru 0xe8035e7 data_size 0x9(0xdc53 bytes) 2024-09-13T04:59:12.944 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.944 INFO:teuthology.orchestra.run.smithi001.stdout:4174 write 0x84f987b thru 0x85057f7 (0xbf7d bytes) 2024-09-13T04:59:12.957 INFO:teuthology.orchestra.run.smithi001.stdout:4175 read 0xa6d59d9 thru 0xa6d6326 (0x94e bytes) 2024-09-13T04:59:12.957 INFO:teuthology.orchestra.run.smithi001.stdout:4176 read 0xb84ebc1 thru 0xb84ee8f (0x2cf bytes) 2024-09-13T04:59:12.958 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.958 INFO:teuthology.orchestra.run.smithi001.stdout:4178 punch from 0x7d3dbe5 to 0x7d4b5cd, (0xd9e8 bytes) 2024-09-13T04:59:12.958 INFO:teuthology.orchestra.run.smithi001.stdout:4179 read 0xdd87634 thru 0xdd89cc8 (0x2695 bytes) 2024-09-13T04:59:12.958 INFO:teuthology.orchestra.run.smithi001.stdout:4182 punch from 0xb9a230c to 0xb9b085d, (0xe551 bytes) 2024-09-13T04:59:12.958 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:12.958 INFO:teuthology.orchestra.run.smithi001.stdout:4186 write 0x1fa5dff thru 0x1fb37f0 (0xd9f2 bytes) 2024-09-13T04:59:12.989 INFO:teuthology.orchestra.run.smithi001.stdout:4187 trunc from 0xe8035e8 to 0x7a9c4fe 2024-09-13T04:59:13.006 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:13.002+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:13.020 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:13.020 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:13.020 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:13.020 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:13.020 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:13.033 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:13.048 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:13.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T04:59:13.049 INFO:teuthology.orchestra.run.smithi001.stdout:4189 write 0xd47007d thru 0xd471908 (0x188c bytes) 2024-09-13T04:59:13.056 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:59:13.062 INFO:teuthology.orchestra.run.smithi001.stdout:4191 writesame 0xbcbbf9f thru 0xbcc05ff data_size 0x1(0x4661 bytes) 2024-09-13T04:59:13.079 INFO:teuthology.orchestra.run.smithi001.stdout:4192 punch from 0xa4a1d9 to 0xa4ea94, (0x48bb bytes) 2024-09-13T04:59:13.079 INFO:teuthology.orchestra.run.smithi001.stdout:4193 read 0x77bb2e0 thru 0x77c9a86 (0xe7a7 bytes) 2024-09-13T04:59:13.091 INFO:teuthology.orchestra.run.smithi001.stdout:4194 writesame 0xdc36375 thru 0xdc372a8 data_size 0xe(0xf34 bytes) 2024-09-13T04:59:13.091 INFO:teuthology.orchestra.run.smithi001.stdout:4196 write 0xacab1d5 thru 0xacb56f2 (0xa51e bytes) 2024-09-13T04:59:13.099 INFO:teuthology.orchestra.run.smithi001.stdout:4198 read 0xa935c5a thru 0xa944f91 (0xf338 bytes) 2024-09-13T04:59:13.099 INFO:teuthology.orchestra.run.smithi001.stdout:4199 writesame 0x9721273 thru 0x972529e data_size 0x4a(0x402c bytes) 2024-09-13T04:59:13.103 INFO:teuthology.orchestra.run.smithi001.stdout:4200 writesame 0xc4e36ce thru 0xc4eb552 data_size 0x7(0x7e85 bytes) 2024-09-13T04:59:13.106 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:13.103+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:13.115 INFO:teuthology.orchestra.run.smithi001.stdout:4201 trunc from 0xdc372a9 to 0xaba8715 2024-09-13T04:59:13.131 INFO:teuthology.orchestra.run.smithi001.stdout:4202 writesame 0x843b81f thru 0x84458a8 data_size 0x2(0xa08a bytes) 2024-09-13T04:59:13.146 INFO:teuthology.orchestra.run.smithi001.stdout:4206 writesame 0x46028f4 thru 0x460cfff data_size 0x2(0xa70c bytes) 2024-09-13T04:59:13.173 INFO:teuthology.orchestra.run.smithi001.stdout:4207 punch from 0xa950b00 to 0xa95194d, (0xe4d bytes) 2024-09-13T04:59:13.173 INFO:teuthology.orchestra.run.smithi001.stdout:4209 read 0x5f5ed27 thru 0x5f6e3f8 (0xf6d2 bytes) 2024-09-13T04:59:13.173 INFO:teuthology.orchestra.run.smithi001.stdout:4211 read 0x714dd5c thru 0x7159d40 (0xbfe5 bytes) 2024-09-13T04:59:13.177 INFO:teuthology.orchestra.run.smithi001.stdout:4212 trunc from 0xaba8715 to 0xd8a0fa5 2024-09-13T04:59:13.197 INFO:teuthology.orchestra.run.smithi001.stdout:4213 write 0xe44f9c7 thru 0xe454e16 (0x5450 bytes) 2024-09-13T04:59:13.202 INFO:teuthology.orchestra.run.smithi001.stdout:4215 read 0x6dbbb47 thru 0x6dc7adf (0xbf99 bytes) 2024-09-13T04:59:13.203 INFO:teuthology.orchestra.run.smithi001.stdout:4216 punch from 0x25c7741 to 0x25cfaab, (0x836a bytes) 2024-09-13T04:59:13.203 INFO:teuthology.orchestra.run.smithi001.stdout:4217 writesame 0xe2e609c thru 0xe2e85b5 data_size 0x3(0x251a bytes) 2024-09-13T04:59:13.207 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:13.207 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:13.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:13.207 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:13.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.207 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:13.207 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.172184231000003, 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:59:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:59:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:59:13.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:59:13.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.212 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.172138594000003, 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.213 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:59:13.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.172090509999997, 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:59:13.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.171994427999998, 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:59:13.222 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:59:13.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.171943057, 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:59:13.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:59:13.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.171897002999998, 2024-09-13T04:59:13.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.171846942000002, 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:59:13.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:59:13.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.240 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 57.054431751999999, 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:59:13.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:59:13.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:59:13.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.963003368999999, 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:59:13.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:13.251 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962979023999999, 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:13.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:59:13.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962897671, 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.256 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:59:13.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.260 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962828141000003, 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:59:13.264 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962800864999998, 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.265 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:59:13.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.269 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962702419999999, 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:13.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:59:13.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:59:13.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962632853000002, 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.274 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962562378999998, 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:59:13.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962536694000001, 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.283 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:13.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.287 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962501506000002, 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962476989999999, 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.292 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:13.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.296 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.962407653, 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:59:13.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:59:13.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:59:13.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.301 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.301 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.301 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.301 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:13.301 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:13.202+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:13.302 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:13.305 INFO:teuthology.orchestra.run.smithi001.stdout:4219 punch from 0x199d3e9 to 0x19a2393, (0x4faa bytes) 2024-09-13T04:59:13.305 INFO:teuthology.orchestra.run.smithi001.stdout:4220 read 0x6ed14fc thru 0x6ed34f7 (0x1ffc bytes) 2024-09-13T04:59:13.305 INFO:teuthology.orchestra.run.smithi001.stdout:4221 writesame 0x7e8c8aa thru 0x7e94bef data_size 0x9(0x8346 bytes) 2024-09-13T04:59:13.305 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.305 INFO:teuthology.orchestra.run.smithi001.stdout:4223 write 0x6de75ef thru 0x6df2241 (0xac53 bytes) 2024-09-13T04:59:13.305 INFO:teuthology.orchestra.run.smithi001.stdout:4224 write 0x9686a18 thru 0x96943cd (0xd9b6 bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4225 read 0x69404d5 thru 0x694bc4d (0xb779 bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4226 read 0x9d9f9bc thru 0x9dae665 (0xecaa bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4227 punch from 0x4acc704 to 0x4ada625, (0xdf21 bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4229 trunc from 0xe454e17 to 0xbaffaad 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4230 read 0x91f4a86 thru 0x91fa2e4 (0x585f bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4231 read 0xb34a45d thru 0xb353129 (0x8ccd bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4232 trunc from 0xbaffaad to 0x788de7a 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4233 read 0x1b62277 thru 0x1b6707b (0x4e05 bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4234 read 0x25d1fa4 thru 0x25e1d1e (0xfd7b bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4235 punch from 0x5ab82f3 to 0x5ac54ed, (0xd1fa bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4236 read 0x6ea4d98 thru 0x6eaf970 (0xabd9 bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4237 write 0x6631643 thru 0x6631bb0 (0x56e bytes) 2024-09-13T04:59:13.306 INFO:teuthology.orchestra.run.smithi001.stdout:4238 read 0x116464e thru 0x116f76d (0xb120 bytes) 2024-09-13T04:59:13.307 INFO:teuthology.orchestra.run.smithi001.stdout:4239 write 0x4f13f8e thru 0x4f1707d (0x30f0 bytes) 2024-09-13T04:59:13.307 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.307 INFO:teuthology.orchestra.run.smithi001.stdout:4241 write 0xac58309 thru 0xac656de (0xd3d6 bytes) 2024-09-13T04:59:13.307 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.307 INFO:teuthology.orchestra.run.smithi001.stdout:4243 read 0x6d2b839 thru 0x6d33891 (0x8059 bytes) 2024-09-13T04:59:13.307 INFO:teuthology.orchestra.run.smithi001.stdout:4244 writesame 0xbe6949 thru 0xbeb245 data_size 0x25(0x48fd bytes) 2024-09-13T04:59:13.307 INFO:teuthology.orchestra.run.smithi001.stdout:4245 write 0x9f63786 thru 0x9f6c58c (0x8e07 bytes) 2024-09-13T04:59:13.311 INFO:teuthology.orchestra.run.smithi001.stdout:4246 read 0x35f0549 thru 0x35fce29 (0xc8e1 bytes) 2024-09-13T04:59:13.311 INFO:teuthology.orchestra.run.smithi001.stdout:4247 read 0x6432fb4 thru 0x643da86 (0xaad3 bytes) 2024-09-13T04:59:13.324 INFO:teuthology.orchestra.run.smithi001.stdout:4248 trunc from 0xac656df to 0x35e56d5 2024-09-13T04:59:13.345 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:13.342+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:13.345 INFO:teuthology.orchestra.run.smithi001.stdout:4249 punch from 0x335b059 to 0x336a132, (0xf0d9 bytes) 2024-09-13T04:59:13.346 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.355 INFO:teuthology.orchestra.run.smithi001.stdout:4252 write 0x3820646 thru 0x3829327 (0x8ce2 bytes) 2024-09-13T04:59:13.360 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.360 INFO:teuthology.orchestra.run.smithi001.stdout:4256 read 0x32de468 thru 0x32e8f8b (0xab24 bytes) 2024-09-13T04:59:13.360 INFO:teuthology.orchestra.run.smithi001.stdout:4257 read 0x3087c5b thru 0x3092aa4 (0xae4a bytes) 2024-09-13T04:59:13.379 INFO:teuthology.orchestra.run.smithi001.stdout:4259 write 0x9b67c7b thru 0x9b73191 (0xb517 bytes) 2024-09-13T04:59:13.384 INFO:teuthology.orchestra.run.smithi001.stdout:4260 read 0x2f210d thru 0x2f92df (0x71d3 bytes) 2024-09-13T04:59:13.385 INFO:teuthology.orchestra.run.smithi001.stdout:4263 read 0x3f7fe1e thru 0x3f8edfa (0xefdd bytes) 2024-09-13T04:59:13.385 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.398 INFO:teuthology.orchestra.run.smithi001.stdout:4266 write 0xc1063f7 thru 0xc112de4 (0xc9ee bytes) 2024-09-13T04:59:13.402 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.409 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:13.406+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:13.414 INFO:teuthology.orchestra.run.smithi001.stdout:4268 write 0xddcffd1 thru 0xddd5bd4 (0x5c04 bytes) 2024-09-13T04:59:13.419 INFO:teuthology.orchestra.run.smithi001.stdout:4269 write 0x3a46bf6 thru 0x3a54984 (0xdd8f bytes) 2024-09-13T04:59:13.430 INFO:teuthology.orchestra.run.smithi001.stdout:4270 read 0xf1226d thru 0xf20e2a (0xebbe bytes) 2024-09-13T04:59:13.435 INFO:teuthology.orchestra.run.smithi001.stdout:4271 trunc from 0xddd5bd5 to 0xa2c5ba4 2024-09-13T04:59:13.453 INFO:teuthology.orchestra.run.smithi001.stdout:4273 punch from 0x9bb8cc4 to 0x9bc7565, (0xe8a1 bytes) 2024-09-13T04:59:13.453 INFO:teuthology.orchestra.run.smithi001.stdout:4274 read 0x3adc469 thru 0x3ae37d6 (0x736e bytes) 2024-09-13T04:59:13.453 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.461 INFO:teuthology.orchestra.run.smithi001.stdout:4276 trunc from 0xa2c5ba4 to 0x600ce7b 2024-09-13T04:59:13.477 INFO:teuthology.orchestra.run.smithi001.stdout:4279 read 0x4b3b15 thru 0x4bb727 (0x7c13 bytes) 2024-09-13T04:59:13.480 INFO:teuthology.orchestra.run.smithi001.stdout:4281 writesame 0x294795d thru 0x2950271 data_size 0x1(0x8915 bytes) 2024-09-13T04:59:13.483 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:13.483 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:13.483 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:13.483 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:13.497 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:13.515 INFO:teuthology.orchestra.run.smithi001.stdout:4282 read 0x54ea7d3 thru 0x54f34f0 (0x8d1e bytes) 2024-09-13T04:59:13.517 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.518 INFO:teuthology.orchestra.run.smithi001.stdout:4284 writesame 0x1d21660 thru 0x1d2db8d data_size 0x5e(0xc52e bytes) 2024-09-13T04:59:13.541 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:13.538+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:13.560 INFO:teuthology.orchestra.run.smithi001.stdout:4286 write 0xcfc59c0 thru 0xcfc93f9 (0x3a3a bytes) 2024-09-13T04:59:13.567 INFO:teuthology.orchestra.run.smithi001.stdout:4287 punch from 0x9f7f1e3 to 0x9f8e12d, (0xef4a bytes) 2024-09-13T04:59:13.568 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.568 INFO:teuthology.orchestra.run.smithi001.stdout:4289 read 0x5a2d1e4 thru 0x5a3551e (0x833b bytes) 2024-09-13T04:59:13.568 INFO:teuthology.orchestra.run.smithi001.stdout:4290 write 0x9e021c2 thru 0x9e0fa2d (0xd86c bytes) 2024-09-13T04:59:13.581 INFO:teuthology.orchestra.run.smithi001.stdout:4291 read 0x739b00c thru 0x73aa89b (0xf890 bytes) 2024-09-13T04:59:13.581 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:13.582 INFO:teuthology.orchestra.run.smithi001.stdout:4293 write 0xb64b04d thru 0xb655cf7 (0xacab bytes) 2024-09-13T04:59:13.590 INFO:teuthology.orchestra.run.smithi001.stdout:4294 write 0x32a030f thru 0x32af2c4 (0xefb6 bytes) 2024-09-13T04:59:13.604 INFO:teuthology.orchestra.run.smithi001.stdout:4295 writesame 0x8d9f33f thru 0x8da1513 data_size 0x1(0x21d5 bytes) 2024-09-13T04:59:13.612 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:13.607+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:13.622 INFO:teuthology.orchestra.run.smithi001.stdout:4296 clone 29 order 24 su 131072 sc 2 2024-09-13T04:59:13.668 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:13.668 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:13.668 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:13.668 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:13.668 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:13.679 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:13.713 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:13.707+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:13.757 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:13.803 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:13.803 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.002162599999998, 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.804 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:13.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.808 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.002128345999999, 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:13.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.002097444999997, 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.002045563999999, 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.002013761000001, 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001979808000002, 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001932072000002, 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001892509999998, 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001844816000002, 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001781733000001, 2024-09-13T04:59:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001745299, 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001711376000003, 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001643162999997, 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001609457000001, 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001559264999997, 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001253232000003, 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001204029999997, 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001155916000002, 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001104439000002, 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.001069358000002, 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:13.898 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:13.900 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:13.900 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:13.894+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:13.917 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:13.934 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:13.935 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:13.950 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:13.967 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:13.984 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:14.001 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:13.995+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:14.002 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:14.002 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:14.020 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:14.039 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:14.039 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T04:59:14.069 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:14.069 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:14.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T04:59:14.085 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:14.132 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:14.130+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:flattening image image_client.0-clone28 2024-09-13T04:59:14.202 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:14.199+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:14.216 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:14.216 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:14.230 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:14.303 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:14.299+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:14.336 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-13T04:59:14.397 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:14.397 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:14.404 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:14.399+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:14.411 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:14.505 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:14.502+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:14.563 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:14.563 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:14.579 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:14.606 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:14.602+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:14.702 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:14.702 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:14.707 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:14.702+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:14.716 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:14.727 INFO:teuthology.orchestra.run.smithi001.stdout:4297 read 0xcf0d38 thru 0xcfd7a4 (0xca6d bytes) 2024-09-13T04:59:14.740 INFO:teuthology.orchestra.run.smithi001.stdout:4298 write 0xe5565a9 thru 0xe55fac1 (0x9519 bytes) 2024-09-13T04:59:14.746 INFO:teuthology.orchestra.run.smithi001.stdout:4299 write 0xbec3ca8 thru 0xbed3b23 (0xfe7c bytes) 2024-09-13T04:59:14.765 INFO:teuthology.orchestra.run.smithi001.stdout:4300 read 0x6fd353d thru 0x6fe0744 (0xd208 bytes) 2024-09-13T04:59:14.765 INFO:teuthology.orchestra.run.smithi001.stdout:4302 read 0x50b3a5a thru 0x50bb817 (0x7dbe bytes) 2024-09-13T04:59:14.765 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:14.765 INFO:teuthology.orchestra.run.smithi001.stdout:4304 writesame 0x5e3e369 thru 0x5e4159d data_size 0x1(0x3235 bytes) 2024-09-13T04:59:14.799 INFO:teuthology.orchestra.run.smithi001.stdout:4305 trunc from 0xe55fac2 to 0x1136928 2024-09-13T04:59:14.808 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:14.803+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:14.854 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:14.854 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:14.870 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:14.887 INFO:teuthology.orchestra.run.smithi001.stdout:4306 write 0x5369b78 thru 0x5374f90 (0xb419 bytes) 2024-09-13T04:59:14.901 INFO:teuthology.orchestra.run.smithi001.stdout:4308 write 0x5cfd0dd thru 0x5cfe0ad (0xfd1 bytes) 2024-09-13T04:59:14.903 INFO:teuthology.orchestra.run.smithi001.stdout:4309 punch from 0x57bde5a to 0x57bf6c3, (0x1869 bytes) 2024-09-13T04:59:14.903 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:14.909 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:14.903+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:14.917 INFO:teuthology.orchestra.run.smithi001.stdout:4312 write 0xa245bd0 thru 0xa247a48 (0x1e79 bytes) 2024-09-13T04:59:14.922 INFO:teuthology.orchestra.run.smithi001.stdout:4313 read 0x523d2f5 thru 0x524ad0b (0xda17 bytes) 2024-09-13T04:59:14.923 INFO:teuthology.orchestra.run.smithi001.stdout:4314 write 0x8e8f556 thru 0x8e9ecb4 (0xf75f bytes) 2024-09-13T04:59:14.936 INFO:teuthology.orchestra.run.smithi001.stdout:4315 trunc from 0xa247a49 to 0xae9a986 2024-09-13T04:59:14.943 INFO:teuthology.orchestra.run.smithi001.stdout:4316 read 0x8508e7c thru 0x8516d47 (0xdecc bytes) 2024-09-13T04:59:14.944 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:14.954 INFO:teuthology.orchestra.run.smithi001.stdout:4318 write 0xbd3ee6b thru 0xbd456ca (0x6860 bytes) 2024-09-13T04:59:14.975 INFO:teuthology.orchestra.run.smithi001.stdout:4319 trunc from 0xbd456cb to 0x10e9b13 2024-09-13T04:59:14.997 INFO:teuthology.orchestra.run.smithi001.stdout:4322 punch from 0xfe26a7 to 0xff24a8, (0xfe01 bytes) 2024-09-13T04:59:14.998 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.010 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:15.007+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:15.024 INFO:teuthology.orchestra.run.smithi001.stdout:4326 writesame 0xb22bf22 thru 0xb236451 data_size 0x18(0xa530 bytes) 2024-09-13T04:59:15.024 INFO:teuthology.orchestra.run.smithi001.stdout:4327 punch from 0x45a08eb to 0x45a944b, (0x8b60 bytes) 2024-09-13T04:59:15.025 INFO:teuthology.orchestra.run.smithi001.stdout:4328 write 0x362e37a thru 0x362e7ac (0x433 bytes) 2024-09-13T04:59:15.030 INFO:teuthology.orchestra.run.smithi001.stdout:4330 read 0x9cf316 thru 0x9dc22f (0xcf1a bytes) 2024-09-13T04:59:15.032 INFO:teuthology.orchestra.run.smithi001.stdout:4332 write 0x869d51c thru 0x86a89a1 (0xb486 bytes) 2024-09-13T04:59:15.032 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:15.032 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:15.043 INFO:teuthology.orchestra.run.smithi001.stdout:4333 read 0x6f4f8c1 thru 0x6f571e2 (0x7922 bytes) 2024-09-13T04:59:15.052 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:15.058 INFO:teuthology.orchestra.run.smithi001.stdout:4335 trunc from 0xb236452 to 0x23bfa07 2024-09-13T04:59:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T04:59:15.081 INFO:teuthology.orchestra.run.smithi001.stdout:4336 punch from 0x142fac7 to 0x143f7c3, (0xfcfc bytes) 2024-09-13T04:59:15.082 INFO:teuthology.orchestra.run.smithi001.stdout:4339 read 0x1e81d06 thru 0x1e8ba0d (0x9d08 bytes) 2024-09-13T04:59:15.082 INFO:teuthology.orchestra.run.smithi001.stdout:4340 read 0x466d2c thru 0x469a38 (0x2d0d bytes) 2024-09-13T04:59:15.082 INFO:teuthology.orchestra.run.smithi001.stdout:4344 read 0x1a7fe2c thru 0x1a85ca4 (0x5e79 bytes) 2024-09-13T04:59:15.083 INFO:teuthology.orchestra.run.smithi001.stdout:4346 punch from 0x1bd268e to 0x1bd9c8f, (0x7601 bytes) 2024-09-13T04:59:15.084 INFO:teuthology.orchestra.run.smithi001.stdout:4347 punch from 0x12357cd to 0x123b5a3, (0x5dd6 bytes) 2024-09-13T04:59:15.084 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.084 INFO:teuthology.orchestra.run.smithi001.stdout:4349 read 0x618e5c thru 0x621293 (0x8438 bytes) 2024-09-13T04:59:15.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T04:59:15.085 INFO:teuthology.orchestra.run.smithi001.stdout:4350 read 0xd9e379 thru 0xda9dd2 (0xba5a bytes) 2024-09-13T04:59:15.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:59:15.105 INFO:teuthology.orchestra.run.smithi001.stdout:4351 write 0xbc468fb thru 0xbc4fdd4 (0x94da bytes) 2024-09-13T04:59:15.110 INFO:teuthology.orchestra.run.smithi001.stdout:truncating to largest ever: 0xee5855f 2024-09-13T04:59:15.114 INFO:teuthology.orchestra.run.smithi001.stdout:4352 trunc from 0xbc4fdd5 to 0xee5855f 2024-09-13T04:59:15.114 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:15.111+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:15.122 INFO:teuthology.orchestra.run.smithi001.stdout:4353 read 0x36c77eb thru 0x36ceeb1 (0x76c7 bytes) 2024-09-13T04:59:15.122 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.122 INFO:teuthology.orchestra.run.smithi001.stdout:4355 read 0xda9a4f3 thru 0xdaa5808 (0xb316 bytes) 2024-09-13T04:59:15.122 INFO:teuthology.orchestra.run.smithi001.stdout:4356 read 0xa22d270 thru 0xa22e00a (0xd9b bytes) 2024-09-13T04:59:15.123 INFO:teuthology.orchestra.run.smithi001.stdout:4357 punch from 0x4a3ab79 to 0x4a440e2, (0x9569 bytes) 2024-09-13T04:59:15.123 INFO:teuthology.orchestra.run.smithi001.stdout:4358 write 0x2a151ce thru 0x2a23064 (0xde97 bytes) 2024-09-13T04:59:15.133 INFO:teuthology.orchestra.run.smithi001.stdout:4359 write 0x3577dd8 thru 0x3579a73 (0x1c9c bytes) 2024-09-13T04:59:15.137 INFO:teuthology.orchestra.run.smithi001.stdout:4360 punch from 0x9c6731d to 0x9c75345, (0xe028 bytes) 2024-09-13T04:59:15.137 INFO:teuthology.orchestra.run.smithi001.stdout:4361 read 0xe24bf7a thru 0xe255405 (0x948c bytes) 2024-09-13T04:59:15.137 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.137 INFO:teuthology.orchestra.run.smithi001.stdout:4363 write 0x236aeb6 thru 0x2370e8f (0x5fda bytes) 2024-09-13T04:59:15.142 INFO:teuthology.orchestra.run.smithi001.stdout:4364 writesame 0x39642fc thru 0x396c883 data_size 0x4(0x8588 bytes) 2024-09-13T04:59:15.152 INFO:teuthology.orchestra.run.smithi001.stdout:4365 write 0x3cf07a3 thru 0x3cfa7d1 (0xa02f bytes) 2024-09-13T04:59:15.159 INFO:teuthology.orchestra.run.smithi001.stdout:4366 write 0x20d7b82 thru 0x20da15a (0x25d9 bytes) 2024-09-13T04:59:15.162 INFO:teuthology.orchestra.run.smithi001.stdout:4370 read 0xe483fc2 thru 0xe4862af (0x22ee bytes) 2024-09-13T04:59:15.167 INFO:teuthology.orchestra.run.smithi001.stdout:4372 trunc from 0xee5855f to 0xbb92edd 2024-09-13T04:59:15.177 INFO:teuthology.orchestra.run.smithi001.stdout:4375 write 0x6ab4c5b thru 0x6ac079a (0xbb40 bytes) 2024-09-13T04:59:15.189 INFO:teuthology.orchestra.run.smithi001.stdout:4376 punch from 0xa15a8fa to 0xa1607fd, (0x5f03 bytes) 2024-09-13T04:59:15.189 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.189 INFO:teuthology.orchestra.run.smithi001.stdout:4380 write 0x37c179c thru 0x37c97f0 (0x8055 bytes) 2024-09-13T04:59:15.198 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:15.198 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:15.205 INFO:teuthology.orchestra.run.smithi001.stdout:4382 writesame 0xd427ed4 thru 0xd42887f data_size 0x2(0x9ac bytes) 2024-09-13T04:59:15.211 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:15.207+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:15.223 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:15.227 INFO:teuthology.orchestra.run.smithi001.stdout:4384 trunc from 0xd428880 to 0x80bd26 2024-09-13T04:59:15.244 INFO:teuthology.orchestra.run.smithi001.stdout:4385 punch from 0x474442 to 0x4815af, (0xd16d bytes) 2024-09-13T04:59:15.244 INFO:teuthology.orchestra.run.smithi001.stdout:4386 punch from 0x72df66 to 0x735035, (0x70cf bytes) 2024-09-13T04:59:15.244 INFO:teuthology.orchestra.run.smithi001.stdout:4387 read 0x5c2269 thru 0x5c6408 (0x41a0 bytes) 2024-09-13T04:59:15.262 INFO:teuthology.orchestra.run.smithi001.stdout:4388 write 0x4ec93cc thru 0x4ed803c (0xec71 bytes) 2024-09-13T04:59:15.268 INFO:teuthology.orchestra.run.smithi001.stdout:4392 trunc from 0x4ed803d to 0x52525c4 2024-09-13T04:59:15.276 INFO:teuthology.orchestra.run.smithi001.stdout:4393 punch from 0x3f7fddb to 0x3f88e38, (0x905d bytes) 2024-09-13T04:59:15.277 INFO:teuthology.orchestra.run.smithi001.stdout:4394 write 0x39e003f thru 0x39ede6e (0xde30 bytes) 2024-09-13T04:59:15.303 INFO:teuthology.orchestra.run.smithi001.stdout:4395 writesame 0xcdec18c thru 0xcdf06eb data_size 0x50(0x4560 bytes) 2024-09-13T04:59:15.304 INFO:teuthology.orchestra.run.smithi001.stdout:4397 read 0x222bd47 thru 0x223b817 (0xfad1 bytes) 2024-09-13T04:59:15.304 INFO:teuthology.orchestra.run.smithi001.stdout:4399 read 0x9142d28 thru 0x914b371 (0x864a bytes) 2024-09-13T04:59:15.312 INFO:teuthology.orchestra.run.smithi001.stdout:4400 writesame 0xe0640b2 thru 0xe067171 data_size 0x60(0x30c0 bytes) 2024-09-13T04:59:15.313 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:15.310+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:15.320 INFO:teuthology.orchestra.run.smithi001.stdout:4401 write 0xec6af42 thru 0xec78467 (0xd526 bytes) 2024-09-13T04:59:15.326 INFO:teuthology.orchestra.run.smithi001.stdout:4402 punch from 0x4dfe7c7 to 0x4e0d8fa, (0xf133 bytes) 2024-09-13T04:59:15.346 INFO:teuthology.orchestra.run.smithi001.stdout:4404 trunc from 0xec78468 to 0xafdae4 2024-09-13T04:59:15.381 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:15.381 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:15.386 INFO:teuthology.orchestra.run.smithi001.stdout:4406 writesame 0xc14e045 thru 0xc156a5a data_size 0x65(0x8a16 bytes) 2024-09-13T04:59:15.386 INFO:teuthology.orchestra.run.smithi001.stdout:4407 write 0x2c35eb6 thru 0x2c43b26 (0xdc71 bytes) 2024-09-13T04:59:15.398 INFO:teuthology.orchestra.run.smithi001.stdout:4408 read 0x1976433 thru 0x1980a18 (0xa5e6 bytes) 2024-09-13T04:59:15.398 INFO:teuthology.orchestra.run.smithi001.stdout:4409 read 0x3e11416 thru 0x3e1ca03 (0xb5ee bytes) 2024-09-13T04:59:15.400 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:15.400 INFO:teuthology.orchestra.run.smithi001.stdout:4410 flatten 2024-09-13T04:59:15.405 INFO:teuthology.orchestra.run.smithi001.stdout:4412 read 0x2ca0b07 thru 0x2ca57cf (0x4cc9 bytes) 2024-09-13T04:59:15.407 INFO:teuthology.orchestra.run.smithi001.stdout:4413 read 0x4d49d7f thru 0x4d50fdf (0x7261 bytes) 2024-09-13T04:59:15.413 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:15.410+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:15.413 INFO:teuthology.orchestra.run.smithi001.stdout:4414 trunc from 0xc156a5b to 0x73a5dd4 2024-09-13T04:59:15.422 INFO:teuthology.orchestra.run.smithi001.stdout:4415 punch from 0x571a15a to 0x5725887, (0xb72d bytes) 2024-09-13T04:59:15.425 INFO:teuthology.orchestra.run.smithi001.stdout:4416 trunc from 0x73a5dd4 to 0x4cf6ff6 2024-09-13T04:59:15.432 INFO:teuthology.orchestra.run.smithi001.stdout:4417 writesame 0x4675a02 thru 0x467ab1b data_size 0x7(0x511a bytes) 2024-09-13T04:59:15.439 INFO:teuthology.orchestra.run.smithi001.stdout:4419 write 0x14375de thru 0x14448c8 (0xd2eb bytes) 2024-09-13T04:59:15.444 INFO:teuthology.orchestra.run.smithi001.stdout:4420 punch from 0xb462f9 to 0xb4f661, (0x9368 bytes) 2024-09-13T04:59:15.444 INFO:teuthology.orchestra.run.smithi001.stdout:4421 write 0xe6f571 thru 0xe72c4b (0x36db bytes) 2024-09-13T04:59:15.462 INFO:teuthology.orchestra.run.smithi001.stdout:4423 writesame 0xa2010c3 thru 0xa20a40d data_size 0x1(0x934b bytes) 2024-09-13T04:59:15.473 INFO:teuthology.orchestra.run.smithi001.stdout:4425 write 0xd723049 thru 0xd72bb80 (0x8b38 bytes) 2024-09-13T04:59:15.486 INFO:teuthology.orchestra.run.smithi001.stdout:4426 writesame 0xe16b2b8 thru 0xe16d8c6 data_size 0x1(0x260f bytes) 2024-09-13T04:59:15.486 INFO:teuthology.orchestra.run.smithi001.stdout:4428 write 0x8843bae thru 0x884e74f (0xaba2 bytes) 2024-09-13T04:59:15.495 INFO:teuthology.orchestra.run.smithi001.stdout:4429 write 0xcee19f8 thru 0xcef0472 (0xea7b bytes) 2024-09-13T04:59:15.514 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:15.510+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:15.524 INFO:teuthology.orchestra.run.smithi001.stdout:4430 trunc from 0xe16d8c7 to 0xae54ba 2024-09-13T04:59:15.540 INFO:teuthology.orchestra.run.smithi001.stdout:4431 punch from 0x58216e to 0x591f33, (0xfdc5 bytes) 2024-09-13T04:59:15.540 INFO:teuthology.orchestra.run.smithi001.stdout:4433 read 0x450108 thru 0x457b14 (0x7a0d bytes) 2024-09-13T04:59:15.541 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:15.541 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:15.541 INFO:teuthology.orchestra.run.smithi001.stdout:4434 read 0x9f61a3 thru 0xa0531a (0xf178 bytes) 2024-09-13T04:59:15.545 INFO:teuthology.orchestra.run.smithi001.stdout:4435 trunc from 0xae54ba to 0x1eb95cf 2024-09-13T04:59:15.554 INFO:teuthology.orchestra.run.smithi001.stdout:4437 read 0xdac7df thru 0xdb6f63 (0xa785 bytes) 2024-09-13T04:59:15.555 INFO:teuthology.orchestra.run.smithi001.stdout:4439 read 0x10b5dd thru 0x11a7da (0xf1fe bytes) 2024-09-13T04:59:15.557 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.557 INFO:teuthology.orchestra.run.smithi001.stdout:4441 read 0x7a9de2 thru 0x7b93a7 (0xf5c6 bytes) 2024-09-13T04:59:15.559 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:15.581 INFO:teuthology.orchestra.run.smithi001.stdout:4442 write 0xa326955 thru 0xa336256 (0xf902 bytes) 2024-09-13T04:59:15.586 INFO:teuthology.orchestra.run.smithi001.stdout:4443 read 0x34217ef thru 0x3424f51 (0x3763 bytes) 2024-09-13T04:59:15.587 INFO:teuthology.orchestra.run.smithi001.stdout:4444 write 0x4e4b1ba thru 0x4e5888a (0xd6d1 bytes) 2024-09-13T04:59:15.594 INFO:teuthology.orchestra.run.smithi001.stdout:4446 read 0x7d3623a thru 0x7d457ab (0xf572 bytes) 2024-09-13T04:59:15.594 INFO:teuthology.orchestra.run.smithi001.stdout:4452 read 0x692819e thru 0x69281b4 (0x17 bytes) 2024-09-13T04:59:15.595 INFO:teuthology.orchestra.run.smithi001.stdout:4453 punch from 0x25e1af5 to 0x25e3152, (0x165d bytes) 2024-09-13T04:59:15.595 INFO:teuthology.orchestra.run.smithi001.stdout:4455 write 0x4158f0c thru 0x415f9bb (0x6ab0 bytes) 2024-09-13T04:59:15.609 INFO:teuthology.orchestra.run.smithi001.stdout:4456 write 0xd2d9cbe thru 0xd2dbb74 (0x1eb7 bytes) 2024-09-13T04:59:15.614 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.614 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.614 INFO:teuthology.orchestra.run.smithi001.stdout:4460 read 0x879fbf9 thru 0x87ad36f (0xd777 bytes) 2024-09-13T04:59:15.615 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:15.611+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:15.626 INFO:teuthology.orchestra.run.smithi001.stdout:4461 trunc from 0xd2dbb75 to 0x5c42791 2024-09-13T04:59:15.654 INFO:teuthology.orchestra.run.smithi001.stdout:4465 write 0xa5ce942 thru 0xa5de671 (0xfd30 bytes) 2024-09-13T04:59:15.660 INFO:teuthology.orchestra.run.smithi001.stdout:4466 write 0x4803f7b thru 0x480b674 (0x76fa bytes) 2024-09-13T04:59:15.663 INFO:teuthology.orchestra.run.smithi001.stdout:4468 read 0x19f98b0 thru 0x1a09621 (0xfd72 bytes) 2024-09-13T04:59:15.664 INFO:teuthology.orchestra.run.smithi001.stdout:4469 read 0x59d2dbc thru 0x59d8cd6 (0x5f1b bytes) 2024-09-13T04:59:15.664 INFO:teuthology.orchestra.run.smithi001.stdout:4471 read 0x7bc671f thru 0x7bcc7c7 (0x60a9 bytes) 2024-09-13T04:59:15.665 INFO:teuthology.orchestra.run.smithi001.stdout:4472 read 0x2d489dd thru 0x2d49598 (0xbbc bytes) 2024-09-13T04:59:15.665 INFO:teuthology.orchestra.run.smithi001.stdout:4473 write 0x9c63f71 thru 0x9c6da0a (0x9a9a bytes) 2024-09-13T04:59:15.673 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.686 INFO:teuthology.orchestra.run.smithi001.stdout:4475 trunc from 0xa5de672 to 0x2d7dbb4 2024-09-13T04:59:15.704 INFO:teuthology.orchestra.run.smithi001.stdout:4476 read 0x2662860 thru 0x2664a4c (0x21ed bytes) 2024-09-13T04:59:15.704 INFO:teuthology.orchestra.run.smithi001.stdout:4477 read 0xc38e56 thru 0xc3dd3d (0x4ee8 bytes) 2024-09-13T04:59:15.708 INFO:teuthology.orchestra.run.smithi001.stdout:4478 trunc from 0x2d7dbb4 to 0x5a1bdd 2024-09-13T04:59:15.716 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:15.711+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:15.720 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:15.720 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:15.721 INFO:teuthology.orchestra.run.smithi001.stdout:4479 punch from 0x2f5b3e to 0x2fc5fc, (0x6abe bytes) 2024-09-13T04:59:15.733 INFO:teuthology.orchestra.run.smithi001.stdout:4480 write 0x1486d79 thru 0x1493e6d (0xd0f5 bytes) 2024-09-13T04:59:15.738 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:15.738 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.738 INFO:teuthology.orchestra.run.smithi001.stdout:4483 punch from 0xa35d32 to 0xa3d05e, (0x732c bytes) 2024-09-13T04:59:15.747 INFO:teuthology.orchestra.run.smithi001.stdout:4484 trunc from 0x1493e6e to 0x6983694 2024-09-13T04:59:15.755 INFO:teuthology.orchestra.run.smithi001.stdout:4485 read 0x254bdcb thru 0x2553579 (0x77af bytes) 2024-09-13T04:59:15.769 INFO:teuthology.orchestra.run.smithi001.stdout:4486 write 0xa700e82 thru 0xa708f3c (0x80bb bytes) 2024-09-13T04:59:15.775 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.775 INFO:teuthology.orchestra.run.smithi001.stdout:4490 read 0x69ac04e thru 0x69ad0a9 (0x105c bytes) 2024-09-13T04:59:15.775 INFO:teuthology.orchestra.run.smithi001.stdout:4491 read 0x62a78ce thru 0x62abb29 (0x425c bytes) 2024-09-13T04:59:15.784 INFO:teuthology.orchestra.run.smithi001.stdout:4492 write 0xbd7a821 thru 0xbd84eb2 (0xa692 bytes) 2024-09-13T04:59:15.789 INFO:teuthology.orchestra.run.smithi001.stdout:4493 write 0xa677127 thru 0xa67f490 (0x836a bytes) 2024-09-13T04:59:15.804 INFO:teuthology.orchestra.run.smithi001.stdout:4495 trunc from 0xbd84eb3 to 0x5506291 2024-09-13T04:59:15.817 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:15.814+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:15.823 INFO:teuthology.orchestra.run.smithi001.stdout:4496 trunc from 0x5506291 to 0x286effd 2024-09-13T04:59:15.833 INFO:teuthology.orchestra.run.smithi001.stdout:4498 punch from 0x1496b12 to 0x149b0bd, (0x45ab bytes) 2024-09-13T04:59:15.846 INFO:teuthology.orchestra.run.smithi001.stdout:4499 trunc from 0x286effd to 0xb9b6e24 2024-09-13T04:59:15.856 INFO:teuthology.orchestra.run.smithi001.stdout:4500 read 0x3536fe7 thru 0x3546fd8 (0xfff2 bytes) 2024-09-13T04:59:15.856 INFO:teuthology.orchestra.run.smithi001.stdout:4501 write 0xabcf5c thru 0xac11df (0x4284 bytes) 2024-09-13T04:59:15.865 INFO:teuthology.orchestra.run.smithi001.stdout:4502 trunc from 0xb9b6e24 to 0xea22b37 2024-09-13T04:59:15.873 INFO:teuthology.orchestra.run.smithi001.stdout:4503 writesame 0x1e34494 thru 0x1e3e822 data_size 0x11(0xa38f bytes) 2024-09-13T04:59:15.880 INFO:teuthology.orchestra.run.smithi001.stdout:4504 read 0x3bfecc0 thru 0x3c01a6d (0x2dae bytes) 2024-09-13T04:59:15.880 INFO:teuthology.orchestra.run.smithi001.stdout:4509 read 0xd064c43 thru 0xd06d00d (0x83cb bytes) 2024-09-13T04:59:15.880 INFO:teuthology.orchestra.run.smithi001.stdout:4510 write 0xa11d309 thru 0xa125ee0 (0x8bd8 bytes) 2024-09-13T04:59:15.887 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:15.887 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:15.890 INFO:teuthology.orchestra.run.smithi001.stdout:4512 writesame 0x3b765cf thru 0x3b8317d data_size 0x5b(0xcbaf bytes) 2024-09-13T04:59:15.906 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:15.907 INFO:teuthology.orchestra.run.smithi001.stdout:4513 trunc from 0xea22b37 to 0x8cb25f1 2024-09-13T04:59:15.924 INFO:teuthology.orchestra.run.smithi001.stdout:4514 writesame 0x42762c thru 0x433a7b data_size 0x48(0xc450 bytes) 2024-09-13T04:59:15.924 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:15.919+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:15.926 INFO:teuthology.orchestra.run.smithi001.stdout:4515 write 0x5f00910 thru 0x5f03526 (0x2c17 bytes) 2024-09-13T04:59:15.931 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.931 INFO:teuthology.orchestra.run.smithi001.stdout:4517 punch from 0x86c3495 to 0x86c7b1a, (0x4685 bytes) 2024-09-13T04:59:15.931 INFO:teuthology.orchestra.run.smithi001.stdout:4518 write 0x346d87 thru 0x34f17e (0x83f8 bytes) 2024-09-13T04:59:15.943 INFO:teuthology.orchestra.run.smithi001.stdout:4519 trunc from 0x8cb25f1 to 0xc98abad 2024-09-13T04:59:15.950 INFO:teuthology.orchestra.run.smithi001.stdout:4521 read 0x6cc464e thru 0x6cc5644 (0xff7 bytes) 2024-09-13T04:59:15.950 INFO:teuthology.orchestra.run.smithi001.stdout:4524 writesame 0x4c4c447 thru 0x4c4df39 data_size 0x1(0x1af3 bytes) 2024-09-13T04:59:15.956 INFO:teuthology.orchestra.run.smithi001.stdout:4525 read 0x74be221 thru 0x74c9afd (0xb8dd bytes) 2024-09-13T04:59:15.956 INFO:teuthology.orchestra.run.smithi001.stdout:4526 write 0x8f510b8 thru 0x8f54aca (0x3a13 bytes) 2024-09-13T04:59:15.970 INFO:teuthology.orchestra.run.smithi001.stdout:4527 trunc from 0xc98abad to 0x786d99d 2024-09-13T04:59:15.983 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.983 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.983 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:15.983 INFO:teuthology.orchestra.run.smithi001.stdout:4532 write 0x55c9238 thru 0x55d3c72 (0xaa3b bytes) 2024-09-13T04:59:15.986 INFO:teuthology.orchestra.run.smithi001.stdout:4533 write 0x711d801 thru 0x7129fda (0xc7da bytes) 2024-09-13T04:59:16.008 INFO:teuthology.orchestra.run.smithi001.stdout:4534 write 0xaa14493 thru 0xaa18428 (0x3f96 bytes) 2024-09-13T04:59:16.012 INFO:teuthology.orchestra.run.smithi001.stdout:4535 read 0x61a8ddd thru 0x61b0434 (0x7658 bytes) 2024-09-13T04:59:16.019 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:16.015+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:16.022 INFO:teuthology.orchestra.run.smithi001.stdout:4536 write 0xc3211c7 thru 0xc3263cc (0x5206 bytes) 2024-09-13T04:59:16.028 INFO:teuthology.orchestra.run.smithi001.stdout:4538 writesame 0x73494b5 thru 0x73532db data_size 0x1(0x9e27 bytes) 2024-09-13T04:59:16.053 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:16.053 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:16.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:16.064 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:59:16.064 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T04:59:16.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:16.078 INFO:teuthology.orchestra.run.smithi001.stdout:4539 trunc from 0xc3263cd to 0x58dc937 2024-09-13T04:59:16.102 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T04:59:16.106 INFO:teuthology.orchestra.run.smithi001.stdout:4540 writesame 0x97d87 thru 0x9f0a0 data_size 0x9(0x731a bytes) 2024-09-13T04:59:16.115 INFO:teuthology.orchestra.run.smithi001.stdout:4541 punch from 0x577319a to 0x57823b2, (0xf218 bytes) 2024-09-13T04:59:16.115 INFO:teuthology.orchestra.run.smithi001.stdout:4542 punch from 0x559362c to 0x559b665, (0x8039 bytes) 2024-09-13T04:59:16.116 INFO:teuthology.orchestra.run.smithi001.stdout:4544 read 0x2df1790 thru 0x2dfd727 (0xbf98 bytes) 2024-09-13T04:59:16.116 INFO:teuthology.orchestra.run.smithi001.stdout:4545 read 0x1a8b89a thru 0x1a97338 (0xba9f bytes) 2024-09-13T04:59:16.117 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:16.117 INFO:teuthology.orchestra.run.smithi001.stdout:4547 read 0x1012656 thru 0x1018cff (0x66aa bytes) 2024-09-13T04:59:16.118 INFO:teuthology.orchestra.run.smithi001.stdout:4548 writesame 0x4f64b3 thru 0x4f79d0 data_size 0x22(0x151e bytes) 2024-09-13T04:59:16.119 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:16.114+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:16.120 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:16.138 INFO:teuthology.orchestra.run.smithi001.stdout:4551 write 0xcbf666f thru 0xcbf6731 (0xc3 bytes) 2024-09-13T04:59:16.144 INFO:teuthology.orchestra.run.smithi001.stdout:4552 read 0xc26bd9e thru 0xc27685e (0xaac1 bytes) 2024-09-13T04:59:16.145 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:16.153 INFO:teuthology.orchestra.run.smithi001.stdout:4555 write 0xd578b7d thru 0xd57b7b7 (0x2c3b bytes) 2024-09-13T04:59:16.156 INFO:teuthology.orchestra.run.smithi001.stdout:4557 write 0x7a88ec7 thru 0x7a929d8 (0x9b12 bytes) 2024-09-13T04:59:16.163 INFO:teuthology.orchestra.run.smithi001.stdout:4558 read 0xa493e3d thru 0xa496085 (0x2249 bytes) 2024-09-13T04:59:16.163 INFO:teuthology.orchestra.run.smithi001.stdout:4559 writesame 0xc5c293f thru 0xc5cda25 data_size 0x1(0xb0e7 bytes) 2024-09-13T04:59:16.196 INFO:teuthology.orchestra.run.smithi001.stdout:4561 punch from 0xb35ce73 to 0xb35e3c0, (0x154d bytes) 2024-09-13T04:59:16.197 INFO:teuthology.orchestra.run.smithi001.stdout:4562 write 0x6cf7d41 thru 0x6cf9862 (0x1b22 bytes) 2024-09-13T04:59:16.207 INFO:teuthology.orchestra.run.smithi001.stdout:4564 writesame 0xc438e6c thru 0xc445e6a data_size 0x1(0xcfff bytes) 2024-09-13T04:59:16.214 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:16.214 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:16.221 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:16.215+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:16.231 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:16.234 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:16.234 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:16.257 INFO:teuthology.orchestra.run.smithi001.stdout:4565 trunc from 0xd57b7b8 to 0xe4bd56a 2024-09-13T04:59:16.258 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:16.258 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:16.264 INFO:teuthology.orchestra.run.smithi001.stdout:4566 writesame 0x5f95d7a thru 0x5f9820b data_size 0x1f(0x2492 bytes) 2024-09-13T04:59:16.267 INFO:teuthology.orchestra.run.smithi001.stdout:4568 punch from 0xcbb1088 to 0xcbbfd64, (0xecdc bytes) 2024-09-13T04:59:16.267 INFO:teuthology.orchestra.run.smithi001.stdout:4569 read 0x4c4272 thru 0x4cd23d (0x8fcc bytes) 2024-09-13T04:59:16.268 INFO:teuthology.orchestra.run.smithi001.stdout:4570 read 0xe22e72f thru 0xe23912a (0xa9fc bytes) 2024-09-13T04:59:16.268 INFO:teuthology.orchestra.run.smithi001.stdout:4572 write 0x9e827e9 thru 0x9e899d8 (0x71f0 bytes) 2024-09-13T04:59:16.276 INFO:teuthology.orchestra.run.smithi001.stdout:4573 writesame 0xb8672a thru 0xb93db7 data_size 0x1d(0xd68e bytes) 2024-09-13T04:59:16.280 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:16.281 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:16.281 INFO:teuthology.orchestra.run.smithi001.stdout:4574 read 0x8f03d9c thru 0x8f11217 (0xd47c bytes) 2024-09-13T04:59:16.283 INFO:teuthology.orchestra.run.smithi001.stdout:4575 read 0xbe2f827 thru 0xbe3c574 (0xcd4e bytes) 2024-09-13T04:59:16.283 INFO:teuthology.orchestra.run.smithi001.stdout:4576 write 0xa9f6c1b thru 0xaa05235 (0xe61b bytes) 2024-09-13T04:59:16.301 INFO:teuthology.orchestra.run.smithi001.stdout:4577 trunc from 0xe4bd56a to 0xa684503 2024-09-13T04:59:16.303 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:16.303 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:16.323 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:16.318+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:16.324 INFO:teuthology.orchestra.run.smithi001.stdout:4578 read 0x478f610 thru 0x4791cd2 (0x26c3 bytes) 2024-09-13T04:59:16.324 INFO:teuthology.orchestra.run.smithi001.stdout:4579 punch from 0xc6f644 to 0xc7d70d, (0xe0c9 bytes) 2024-09-13T04:59:16.324 INFO:teuthology.orchestra.run.smithi001.stdout:4580 punch from 0x2f9fe50 to 0x2fac778, (0xc928 bytes) 2024-09-13T04:59:16.324 INFO:teuthology.orchestra.run.smithi001.stdout:4581 punch from 0x5a47078 to 0x5a4a435, (0x33bd bytes) 2024-09-13T04:59:16.325 INFO:teuthology.orchestra.run.smithi001.stdout:4584 writesame 0x3fa9d6c thru 0x3fb538f data_size 0x2(0xb624 bytes) 2024-09-13T04:59:16.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:16.327 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:16.340 INFO:teuthology.orchestra.run.smithi001.stdout:4586 write 0x1f4285f thru 0x1f44a96 (0x2238 bytes) 2024-09-13T04:59:16.344 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:16.344 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:16.344 INFO:teuthology.orchestra.run.smithi001.stdout:4590 read 0x13bdc81 thru 0x13c8b72 (0xaef2 bytes) 2024-09-13T04:59:16.344 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:16.345 INFO:teuthology.orchestra.run.smithi001.stdout:4594 write 0x586016 thru 0x587495 (0x1480 bytes) 2024-09-13T04:59:16.348 INFO:teuthology.orchestra.run.smithi001.stdout:4595 write 0x3d48d19 thru 0x3d56d5e (0xe046 bytes) 2024-09-13T04:59:16.351 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:16.351 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:16.365 INFO:teuthology.orchestra.run.smithi001.stdout:4596 write 0xb21615d thru 0xb22308c (0xcf30 bytes) 2024-09-13T04:59:16.368 INFO:teuthology.orchestra.run.smithi001.stdout:4597 read 0x1ea7fe0 thru 0x1ea893b (0x95c bytes) 2024-09-13T04:59:16.368 INFO:teuthology.orchestra.run.smithi001.stdout:4598 writesame 0xa953532 thru 0xa95c973 data_size 0x7(0x9442 bytes) 2024-09-13T04:59:16.372 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.207678627999996, 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:59:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:59:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 60.205237969000002, 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:59:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:59:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:18956 3.11 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30476728~36680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.916005675000001, 2024-09-13T04:59:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053972842, 2024-09-13T04:59:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 18956 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455651+0000", 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.455650+0000", 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459744+0000", 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040938759999999998 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459770+0000", 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6239000000000001e-05 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459779+0000", 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2089999999999994e-06 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459831+0000", 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1901000000000003e-05 2024-09-13T04:59:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.459936+0000", 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105095 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 2", 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.460613+0000", 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067677700000000004 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509128+0000", 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048515133000000002 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509534+0000", 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040543300000000002 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509581+0000", 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7283000000000001e-05 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.509624+0000", 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3250000000000001e-05 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.358406027999997, 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:59:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:59:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.343894976000001, 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:59:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:59:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70501 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 901120~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104846982000002, 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048712395999999998, 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70501 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266810+0000", 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266809+0000", 2024-09-13T04:59:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266812+0000", 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3970000000000001e-06 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266816+0000", 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.729e-06 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266820+0000", 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0690000000000003e-06 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315355+0000", 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048534985000000003 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315417+0000", 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1765000000000002e-05 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315522+0000", 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105142 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70502 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 905216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104811246000001, 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048974827999999998, 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70502 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266846+0000", 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266845+0000", 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266851+0000", 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266857+0000", 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7180000000000001e-06 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266861+0000", 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6569999999999996e-06 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315576+0000", 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048714396 2024-09-13T04:59:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315687+0000", 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111118 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315821+0000", 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013375400000000001 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70503 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 860160~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104758111999999, 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049096338000000003, 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70503 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266899+0000", 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266898+0000", 2024-09-13T04:59:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266901+0000", 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266906+0000", 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4600000000000002e-06 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266911+0000", 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7570000000000004e-06 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315859+0000", 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.048947955000000001 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315920+0000", 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0906000000000003e-05 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.315995+0000", 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5185000000000006e-05 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70504 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 864256~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104742485999999, 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049340255, 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70504 2024-09-13T04:59:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266914+0000", 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266917+0000", 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7180000000000001e-06 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266922+0000", 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.028e-06 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266926+0000", 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4750000000000004e-06 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316035+0000", 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049109070999999997 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316136+0000", 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010061 2024-09-13T04:59:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316255+0000", 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118884 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70505 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 868352~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104727150999999, 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049486095000000001, 2024-09-13T04:59:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70505 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266930+0000", 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266929+0000", 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266932+0000", 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.306e-06 2024-09-13T04:59:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266936+0000", 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8230000000000002e-06 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266941+0000", 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.279e-06 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316291+0000", 2024-09-13T04:59:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049349968000000001 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316344+0000", 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3659999999999998e-05 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316416+0000", 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1606999999999994e-05 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70506 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 856064~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104712668000001, 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049736427, 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70506 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266944+0000", 2024-09-13T04:59:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266946+0000", 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3240000000000001e-06 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266951+0000", 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7489999999999998e-06 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266956+0000", 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9180000000000002e-06 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316460+0000", 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049503783000000003 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316555+0000", 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316681+0000", 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125513 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70507 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 872448~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104682531999998, 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049868537999999997, 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70507 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266974+0000", 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266977+0000", 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8459999999999999e-06 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266982+0000", 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3179999999999997e-06 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.266986+0000", 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4320000000000001e-06 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316715+0000", 2024-09-13T04:59:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049728886 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316772+0000", 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6805000000000002e-05 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316843+0000", 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0928999999999998e-05 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70508 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 909312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104655339000001, 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050098392999999998, 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70508 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267002+0000", 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267001+0000", 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267004+0000", 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2589999999999999e-06 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267009+0000", 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4410000000000001e-06 2024-09-13T04:59:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267014+0000", 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6319999999999999e-06 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316884+0000", 2024-09-13T04:59:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.049869956 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.316982+0000", 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8472000000000002e-05 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317100+0000", 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117977 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70509 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 876544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104639928000001, 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050242336999999998, 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70509 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267017+0000", 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267019+0000", 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1969999999999999e-06 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267024+0000", 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8029999999999996e-06 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267028+0000", 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2169999999999996e-06 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317133+0000", 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050105246999999999 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317190+0000", 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6623000000000003e-05 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317259+0000", 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9599999999999998e-05 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70510 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 913408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104625374000001, 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050487833000000003, 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70510 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267032+0000", 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267031+0000", 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267033+0000", 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5660000000000001e-06 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267038+0000", 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7650000000000001e-06 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267042+0000", 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.233e-06 2024-09-13T04:59:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317301+0000", 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050258540999999997 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317403+0000", 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101727 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317519+0000", 2024-09-13T04:59:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116685 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70511 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 884736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104611213999998, 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050628583999999997, 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70511 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267046+0000", 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267045+0000", 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267048+0000", 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1430000000000001e-06 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267054+0000", 2024-09-13T04:59:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7950000000000002e-06 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267059+0000", 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.318e-06 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317553+0000", 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050494202000000002 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317605+0000", 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2039000000000001e-05 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317674+0000", 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9419999999999996e-05 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70512 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 880640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104559403, 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050833717, 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70512 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267098+0000", 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267097+0000", 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267100+0000", 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7630000000000001e-06 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267105+0000", 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2850000000000002e-06 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267109+0000", 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5070000000000002e-06 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317716+0000", 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050606774 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317810+0000", 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3851000000000007e-05 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317931+0000", 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121177 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70513 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 892928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104543700000001, 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050973768000000003, 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70513 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267113+0000", 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267115+0000", 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2139999999999999e-06 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267122+0000", 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9639999999999997e-06 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267127+0000", 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4789999999999998e-06 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.317967+0000", 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050840242000000001 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318019+0000", 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.189e-05 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318087+0000", 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8312999999999995e-05 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70514 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104526626999998, 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051225729999999997, 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70514 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267130+0000", 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267136+0000", 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1399999999999997e-06 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267141+0000", 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0690000000000001e-06 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267145+0000", 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3830000000000002e-06 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318126+0000", 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050980481000000001 2024-09-13T04:59:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318237+0000", 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110851 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318356+0000", 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000119499 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:70515 3.d 3:b164a247:::rbd_data.112a82343b36.0000000000000001:11 [read 897024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.104474591999999, 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051324634000000001, 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 70515 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267182+0000", 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267184+0000", 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6529999999999998e-06 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267190+0000", 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.267194+0000", 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4859999999999999e-06 2024-09-13T04:59:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318381+0000", 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.051186785999999998 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318436+0000", 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5180999999999999e-05 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:49.318507+0000", 2024-09-13T04:59:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0753999999999994e-05 2024-09-13T04:59:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:16.472 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:16.473 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:16.473 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:16.474 INFO:teuthology.orchestra.run.smithi001.stdout:4599 writesame 0x663c6d8 thru 0x6645a4f data_size 0x16(0x9378 bytes) 2024-09-13T04:59:16.474 INFO:teuthology.orchestra.run.smithi001.stdout:4601 punch from 0x91a80fd to 0x91b6933, (0xe836 bytes) 2024-09-13T04:59:16.474 INFO:teuthology.orchestra.run.smithi001.stdout:4602 read 0xaf4a671 thru 0xaf58019 (0xd9a9 bytes) 2024-09-13T04:59:16.474 INFO:teuthology.orchestra.run.smithi001.stdout:4603 writesame 0x6a20fac thru 0x6a2efd2 data_size 0x1(0xe027 bytes) 2024-09-13T04:59:16.475 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:16.475 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:16.476 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:4605 read 0x4ea9940 thru 0x4eae4a9 (0x4b6a bytes) 2024-09-13T04:59:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:4606 punch from 0xa7cc89 to 0xa8a621, (0xd998 bytes) 2024-09-13T04:59:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:4607 writesame 0x2adf2d thru 0x2b196c data_size 0x20(0x3a40 bytes) 2024-09-13T04:59:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:4608 trunc from 0xb22308d to 0x7c74798 2024-09-13T04:59:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:4609 punch from 0x48af7bb to 0x48b27ea, (0x302f bytes) 2024-09-13T04:59:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:16.479 INFO:teuthology.orchestra.run.smithi001.stdout:4611 read 0x196e475 thru 0x197b016 (0xcba2 bytes) 2024-09-13T04:59:16.479 INFO:teuthology.orchestra.run.smithi001.stdout:4614 write 0xb41b79d thru 0xb41d850 (0x20b4 bytes) 2024-09-13T04:59:16.479 INFO:teuthology.orchestra.run.smithi001.stdout:4615 read 0x140eeb1 thru 0x14151d5 (0x6325 bytes) 2024-09-13T04:59:16.479 INFO:teuthology.orchestra.run.smithi001.stdout:4616 read 0xf4ca49 thru 0xf5863f (0xbbf7 bytes) 2024-09-13T04:59:16.479 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:16.471+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:16.479 INFO:teuthology.orchestra.run.smithi001.stdout:4617 read 0x3695f9e thru 0x36983ae (0x2411 bytes) 2024-09-13T04:59:16.480 INFO:teuthology.orchestra.run.smithi001.stdout:4618 read 0x7d54012 thru 0x7d58828 (0x4817 bytes) 2024-09-13T04:59:16.480 INFO:teuthology.orchestra.run.smithi001.stdout:4619 punch from 0x5f6f3e7 to 0x5f7d20b, (0xde24 bytes) 2024-09-13T04:59:16.490 INFO:teuthology.orchestra.run.smithi001.stdout:4621 trunc from 0xb41d851 to 0x2247232 2024-09-13T04:59:16.506 INFO:teuthology.orchestra.run.smithi001.stdout:4622 punch from 0xf6099e to 0xf61f8e, (0x15f0 bytes) 2024-09-13T04:59:16.514 INFO:teuthology.orchestra.run.smithi001.stdout:4623 write 0x2f756f8 thru 0x2f83674 (0xdf7d bytes) 2024-09-13T04:59:16.517 INFO:teuthology.orchestra.run.smithi001.stdout:4626 writesame 0x26ea441 thru 0x26fa284 data_size 0x2(0xfe44 bytes) 2024-09-13T04:59:16.579 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:16.575+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:16.581 INFO:teuthology.orchestra.run.smithi001.stdout:4627 writesame 0x9406bb8 thru 0x940a0a3 data_size 0x4(0x34ec bytes) 2024-09-13T04:59:16.591 INFO:teuthology.orchestra.run.smithi001.stdout:4628 trunc from 0x940a0a4 to 0xd68dfc3 2024-09-13T04:59:16.605 INFO:teuthology.orchestra.run.smithi001.stdout:4629 read 0x27232cf thru 0x2730405 (0xd137 bytes) 2024-09-13T04:59:16.606 INFO:teuthology.orchestra.run.smithi001.stdout:4630 read 0xbe55d14 thru 0xbe56764 (0xa51 bytes) 2024-09-13T04:59:16.606 INFO:teuthology.orchestra.run.smithi001.stdout:4633 read 0x63110d1 thru 0x6314e81 (0x3db1 bytes) 2024-09-13T04:59:16.606 INFO:teuthology.orchestra.run.smithi001.stdout:4634 writesame 0xa79b290 thru 0xa79fe72 data_size 0x1(0x4be3 bytes) 2024-09-13T04:59:16.610 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:16.610 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:16.610 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:16.611 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:16.625 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:16.656 INFO:teuthology.orchestra.run.smithi001.stdout:4636 writesame 0xe0d5e22 thru 0xe0e3859 data_size 0x4(0xda38 bytes) 2024-09-13T04:59:16.656 INFO:teuthology.orchestra.run.smithi001.stdout:4637 read 0xd55abc3 thru 0xd5653d8 (0xa816 bytes) 2024-09-13T04:59:16.656 INFO:teuthology.orchestra.run.smithi001.stdout:4639 read 0x7127c46 thru 0x712ea9d (0x6e58 bytes) 2024-09-13T04:59:16.668 INFO:teuthology.orchestra.run.smithi001.stdout:4640 trunc from 0xe0e385a to 0x9de3c02 2024-09-13T04:59:16.679 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:16.675+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:16.709 INFO:teuthology.orchestra.run.smithi001.stdout:4641 trunc from 0x9de3c02 to 0x5117bdb 2024-09-13T04:59:16.722 INFO:teuthology.orchestra.run.smithi001.stdout:4643 punch from 0x9a837f to 0x9aa05c, (0x1cdd bytes) 2024-09-13T04:59:16.736 INFO:teuthology.orchestra.run.smithi001.stdout:4645 trunc from 0x5117bdb to 0xa1f7157 2024-09-13T04:59:16.751 INFO:teuthology.orchestra.run.smithi001.stdout:4646 punch from 0x7e9dd0c to 0x7ea8ef5, (0xb1e9 bytes) 2024-09-13T04:59:16.751 INFO:teuthology.orchestra.run.smithi001.stdout:4647 read 0x82738ed thru 0x827f7ef (0xbf03 bytes) 2024-09-13T04:59:16.757 INFO:teuthology.orchestra.run.smithi001.stdout:4648 trunc from 0xa1f7157 to 0xc75ad90 2024-09-13T04:59:16.761 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:16.761 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:16.761 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:16.761 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:16.761 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:16.771 INFO:teuthology.orchestra.run.smithi001.stdout:4649 read 0x5441781 thru 0x54443e5 (0x2c65 bytes) 2024-09-13T04:59:16.771 INFO:teuthology.orchestra.run.smithi001.stdout:4650 writesame 0x1f0e01c thru 0x1f16e29 data_size 0xb(0x8e0e bytes) 2024-09-13T04:59:16.774 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:16.778 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:16.775+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:16.780 INFO:teuthology.orchestra.run.smithi001.stdout:4652 read 0x4c06b12 thru 0x4c13207 (0xc6f6 bytes) 2024-09-13T04:59:16.780 INFO:teuthology.orchestra.run.smithi001.stdout:4656 write 0x354fd30 thru 0x35556a8 (0x5979 bytes) 2024-09-13T04:59:16.785 INFO:teuthology.orchestra.run.smithi001.stdout:4657 write 0x34c52f4 thru 0x34c6680 (0x138d bytes) 2024-09-13T04:59:16.788 INFO:teuthology.orchestra.run.smithi001.stdout:4658 write 0x81032c8 thru 0x810819e (0x4ed7 bytes) 2024-09-13T04:59:16.811 INFO:teuthology.orchestra.run.smithi001.stdout:4661 trunc from 0xc75ad90 to 0x7c153f8 2024-09-13T04:59:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:4663 read 0x46c696 thru 0x471c37 (0x55a2 bytes) 2024-09-13T04:59:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:4664 read 0x50223f3 thru 0x502a55e (0x816c bytes) 2024-09-13T04:59:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:4665 trunc from 0x7c153f8 to 0x7cbeb95 2024-09-13T04:59:16.872 INFO:teuthology.orchestra.run.smithi001.stdout:4666 writesame 0xb4c8f51 thru 0xb4d0e03 data_size 0xd(0x7eb3 bytes) 2024-09-13T04:59:16.879 INFO:teuthology.orchestra.run.smithi001.stdout:4667 trunc from 0xb4d0e04 to 0x8f82d64 2024-09-13T04:59:16.880 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:16.875+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:16.894 INFO:teuthology.orchestra.run.smithi001.stdout:4668 write 0x1e668f9 thru 0x1e67307 (0xa0f bytes) 2024-09-13T04:59:16.897 INFO:teuthology.orchestra.run.smithi001.stdout:4669 punch from 0x52e7168 to 0x52e829a, (0x1132 bytes) 2024-09-13T04:59:16.914 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:16.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.049968335999999, 2024-09-13T04:59:16.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T04:59:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T04:59:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.916 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T04:59:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T04:59:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T04:59:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T04:59:16.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T04:59:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T04:59:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:16.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T04:59:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T04:59:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T04:59:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T04:59:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T04:59:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T04:59:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.922 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.049941425, 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.923 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.927 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.046195788999999, 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T04:59:16.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:16.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T04:59:16.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.932 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.042652464, 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.933 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T04:59:16.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T04:59:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.042612836, 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.938 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:16.943 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.042589176, 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T04:59:16.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.948 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.042501627, 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T04:59:16.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T04:59:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T04:59:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T04:59:16.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.042448970999999, 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.954 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:16.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T04:59:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T04:59:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T04:59:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T04:59:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T04:59:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T04:59:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:16.960 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.038523080000001, 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T04:59:16.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.038497455000002, 2024-09-13T04:59:16.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T04:59:16.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T04:59:16.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.970 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.038464060999999, 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T04:59:16.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.033674132000002, 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.977 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T04:59:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:16.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.030279502999999, 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.982 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T04:59:16.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T04:59:16.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T04:59:16.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T04:59:16.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T04:59:16.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.030206827000001, 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.989 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:16.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T04:59:16.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T04:59:16.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.026464466, 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:16.994 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:16.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T04:59:16.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T04:59:16.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T04:59:16.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:16.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:16.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:16.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T04:59:16.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T04:59:16.999 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:16.999 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.026291475000001, 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.000 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T04:59:17.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T04:59:17.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 23.022448875999999, 2024-09-13T04:59:17.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:17.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T04:59:17.009 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.982600898000001, 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.010 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:17.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:17.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T04:59:17.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.015 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.978271675999999, 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.016 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.018 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.020 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 22.970542676000001, 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.021 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.022 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.022 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.022 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.022 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T04:59:17.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.022 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:17.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:17.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T04:59:17.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:17.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T04:59:17.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T04:59:17.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T04:59:17.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T04:59:17.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T04:59:17.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T04:59:17.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T04:59:17.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T04:59:17.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T04:59:17.028 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.028 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.028 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.028 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.028 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.029 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:17.029 INFO:teuthology.orchestra.run.smithi001.stdout:4670 write 0xb0d2086 thru 0xb0d2934 (0x8af bytes) 2024-09-13T04:59:17.029 INFO:teuthology.orchestra.run.smithi001.stdout:4671 write 0x3584610 thru 0x358d02a (0x8a1b bytes) 2024-09-13T04:59:17.029 INFO:teuthology.orchestra.run.smithi001.stdout:4672 read 0x42d326c thru 0x42d54b4 (0x2249 bytes) 2024-09-13T04:59:17.029 INFO:teuthology.orchestra.run.smithi001.stdout:4673 read 0x2305dec thru 0x230fa0c (0x9c21 bytes) 2024-09-13T04:59:17.029 INFO:teuthology.orchestra.run.smithi001.stdout:4674 writesame 0x913b559 thru 0x913d078 data_size 0x20(0x1b20 bytes) 2024-09-13T04:59:17.029 INFO:teuthology.orchestra.run.smithi001.stdout:4675 write 0x9f78762 thru 0x9f854c2 (0xcd61 bytes) 2024-09-13T04:59:17.030 INFO:teuthology.orchestra.run.smithi001.stdout:4676 trunc from 0xb0d2935 to 0xafb4c36 2024-09-13T04:59:17.031 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:17.031 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:16.983+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:17.032 INFO:teuthology.orchestra.run.smithi001.stdout:4677 trunc from 0xafb4c36 to 0x51e3f77 2024-09-13T04:59:17.032 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.042 INFO:teuthology.orchestra.run.smithi001.stdout:4679 write 0x9dd67f8 thru 0x9de1a63 (0xb26c bytes) 2024-09-13T04:59:17.050 INFO:teuthology.orchestra.run.smithi001.stdout:4680 read 0x2e2780e thru 0x2e2f5c9 (0x7dbc bytes) 2024-09-13T04:59:17.051 INFO:teuthology.orchestra.run.smithi001.stdout:4681 writesame 0xf8ed14 thru 0xf9113b data_size 0x34(0x2428 bytes) 2024-09-13T04:59:17.054 INFO:teuthology.orchestra.run.smithi001.stdout:4682 read 0x5fd687f thru 0x5fdc5fe (0x5d80 bytes) 2024-09-13T04:59:17.054 INFO:teuthology.orchestra.run.smithi001.stdout:4683 write 0x901a65b thru 0x901ba73 (0x1419 bytes) 2024-09-13T04:59:17.074 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T04:59:17.075 INFO:teuthology.orchestra.run.smithi001.stdout:4684 write 0xa09c51d thru 0xa09f8f0 (0x33d4 bytes) 2024-09-13T04:59:17.083 INFO:teuthology.orchestra.run.smithi001.stdout:4685 read 0x9a4ef7e thru 0x9a5c3fd (0xd480 bytes) 2024-09-13T04:59:17.086 INFO:teuthology.orchestra.run.smithi001.stdout:4686 punch from 0xa007c5c to 0xa00f4d4, (0x7878 bytes) 2024-09-13T04:59:17.088 INFO:teuthology.orchestra.run.smithi001.stdout:4687 trunc from 0xa09f8f1 to 0xb561dde 2024-09-13T04:59:17.088 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:17.086+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:17.097 INFO:teuthology.orchestra.run.smithi001.stdout:4688 write 0x82224c7 thru 0x822f806 (0xd340 bytes) 2024-09-13T04:59:17.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T04:59:17.107 INFO:teuthology.orchestra.run.smithi001.stdout:4689 trunc from 0xb561dde to 0xdc3abff 2024-09-13T04:59:17.116 INFO:teuthology.orchestra.run.smithi001.stdout:4690 read 0xc7e65f thru 0xc800fc (0x1a9e bytes) 2024-09-13T04:59:17.120 INFO:teuthology.orchestra.run.smithi001.stdout:4696 punch from 0xb5dd038 to 0xb5e0a34, (0x39fc bytes) 2024-09-13T04:59:17.120 INFO:teuthology.orchestra.run.smithi001.stdout:4697 writesame 0xa59a823 thru 0xa59d7f9 data_size 0x1(0x2fd7 bytes) 2024-09-13T04:59:17.130 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.130 INFO:teuthology.orchestra.run.smithi001.stdout:4700 punch from 0x4ad82a5 to 0x4addd23, (0x5a7e bytes) 2024-09-13T04:59:17.131 INFO:teuthology.orchestra.run.smithi001.stdout:4702 punch from 0x1ea5f5e to 0x1ea6d51, (0xdf3 bytes) 2024-09-13T04:59:17.131 INFO:teuthology.orchestra.run.smithi001.stdout:4703 read 0x5f1585a thru 0x5f256be (0xfe65 bytes) 2024-09-13T04:59:17.131 INFO:teuthology.orchestra.run.smithi001.stdout:4705 read 0x4d18bb6 thru 0x4d1d64c (0x4a97 bytes) 2024-09-13T04:59:17.131 INFO:teuthology.orchestra.run.smithi001.stdout:4706 read 0xdffab1 thru 0xe0360a (0x3b5a bytes) 2024-09-13T04:59:17.131 INFO:teuthology.orchestra.run.smithi001.stdout:4707 writesame 0xc5d9d71 thru 0xc5e6b64 data_size 0x1(0xcdf4 bytes) 2024-09-13T04:59:17.166 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:17.166 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:17.166 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:17.166 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:17.166 INFO:teuthology.orchestra.run.smithi001.stdout:4710 write 0xb24dcdb thru 0xb2577b7 (0x9add bytes) 2024-09-13T04:59:17.173 INFO:teuthology.orchestra.run.smithi001.stdout:4711 read 0x4238d30 thru 0x4243ff0 (0xb2c1 bytes) 2024-09-13T04:59:17.173 INFO:teuthology.orchestra.run.smithi001.stdout:4712 write 0xb342668 thru 0xb350b74 (0xe50d bytes) 2024-09-13T04:59:17.178 INFO:teuthology.orchestra.run.smithi001.stdout:4713 writesame 0xaf2d33e thru 0xaf3539e data_size 0x7(0x8061 bytes) 2024-09-13T04:59:17.183 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:4714 read 0x7b3dd6c thru 0x7b447ad (0x6a42 bytes) 2024-09-13T04:59:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:4715 read 0x8087292 thru 0x8089f64 (0x2cd3 bytes) 2024-09-13T04:59:17.195 INFO:teuthology.orchestra.run.smithi001.stdout:4718 write 0xe124b74 thru 0xe127aee (0x2f7b bytes) 2024-09-13T04:59:17.200 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.200 INFO:teuthology.orchestra.run.smithi001.stdout:4721 read 0x83476f3 thru 0x8349635 (0x1f43 bytes) 2024-09-13T04:59:17.201 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.201 INFO:teuthology.orchestra.run.smithi001.stdout:4723 writesame 0x5df1349 thru 0x5df4725 data_size 0x1(0x33dd bytes) 2024-09-13T04:59:17.212 INFO:teuthology.orchestra.run.smithi001.stdout:4724 punch from 0x4e4c318 to 0x4e4ea75, (0x275d bytes) 2024-09-13T04:59:17.212 INFO:teuthology.orchestra.run.smithi001.stdout:4725 punch from 0xa83cbdb to 0xa842d5b, (0x6180 bytes) 2024-09-13T04:59:17.212 INFO:teuthology.orchestra.run.smithi001.stdout:4726 write 0xaef4646 thru 0xaefa42b (0x5de6 bytes) 2024-09-13T04:59:17.215 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.225 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:17.222+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:17.226 INFO:teuthology.orchestra.run.smithi001.stdout:4729 trunc from 0xe127aef to 0x6c6527a 2024-09-13T04:59:17.258 INFO:teuthology.orchestra.run.smithi001.stdout:4730 trunc from 0x6c6527a to 0xed72ff3 2024-09-13T04:59:17.264 INFO:teuthology.orchestra.run.smithi001.stdout:4732 write 0xe2ec5ec thru 0xe2fbb02 (0xf517 bytes) 2024-09-13T04:59:17.272 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.272 INFO:teuthology.orchestra.run.smithi001.stdout:4736 write 0x93f0aab thru 0x93fad5d (0xa2b3 bytes) 2024-09-13T04:59:17.278 INFO:teuthology.orchestra.run.smithi001.stdout:4737 punch from 0xcf6374a to 0xcf6f464, (0xbd1a bytes) 2024-09-13T04:59:17.279 INFO:teuthology.orchestra.run.smithi001.stdout:4738 read 0x37658d2 thru 0x376d1e7 (0x7916 bytes) 2024-09-13T04:59:17.279 INFO:teuthology.orchestra.run.smithi001.stdout:4739 write 0x900b68c thru 0x9012ae5 (0x745a bytes) 2024-09-13T04:59:17.285 INFO:teuthology.orchestra.run.smithi001.stdout:4740 writesame 0x335bd23 thru 0x3363252 data_size 0x50(0x7530 bytes) 2024-09-13T04:59:17.288 INFO:teuthology.orchestra.run.smithi001.stdout:4741 writesame 0x241b006 thru 0x241cfa9 data_size 0x51(0x1fa4 bytes) 2024-09-13T04:59:17.289 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:17.286+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:17.290 INFO:teuthology.orchestra.run.smithi001.stdout:4742 read 0xe10f412 thru 0xe111b2f (0x271e bytes) 2024-09-13T04:59:17.290 INFO:teuthology.orchestra.run.smithi001.stdout:4743 write 0x8ddc24a thru 0x8dddb60 (0x1917 bytes) 2024-09-13T04:59:17.291 INFO:teuthology.orchestra.run.smithi001.stdout:4745 writesame 0x96ce319 thru 0x96d28e1 data_size 0xf(0x45c9 bytes) 2024-09-13T04:59:17.294 INFO:teuthology.orchestra.run.smithi001.stdout:4746 writesame 0x84b6bb9 thru 0x84bf9ed data_size 0xf(0x8e35 bytes) 2024-09-13T04:59:17.298 INFO:teuthology.orchestra.run.smithi001.stdout:4747 writesame 0xc5164b thru 0xc5c85a data_size 0x70(0xb210 bytes) 2024-09-13T04:59:17.301 INFO:teuthology.orchestra.run.smithi001.stdout:4748 write 0x45a92da thru 0x45b6f75 (0xdc9c bytes) 2024-09-13T04:59:17.306 INFO:teuthology.orchestra.run.smithi001.stdout:4749 writesame 0xe543612 thru 0xe54ecf4 data_size 0x1(0xb6e3 bytes) 2024-09-13T04:59:17.336 INFO:teuthology.orchestra.run.smithi001.stdout:4752 writesame 0x19602fa thru 0x196f250 data_size 0x1(0xef57 bytes) 2024-09-13T04:59:17.351 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:17.351 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:17.351 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:17.351 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:17.351 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:17.365 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:17.384 INFO:teuthology.orchestra.run.smithi001.stdout:4753 read 0x4aa49e5 thru 0x4aab08b (0x66a7 bytes) 2024-09-13T04:59:17.384 INFO:teuthology.orchestra.run.smithi001.stdout:4754 write 0xcf73221 thru 0xcf7ecea (0xbaca bytes) 2024-09-13T04:59:17.390 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:17.387+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:17.397 INFO:teuthology.orchestra.run.smithi001.stdout:4755 write 0x124a846 thru 0x124aae6 (0x2a1 bytes) 2024-09-13T04:59:17.399 INFO:teuthology.orchestra.run.smithi001.stdout:4756 punch from 0x296519f to 0x29710df, (0xbf40 bytes) 2024-09-13T04:59:17.399 INFO:teuthology.orchestra.run.smithi001.stdout:4757 writesame 0xd05e4b9 thru 0xd05e824 data_size 0x4(0x36c bytes) 2024-09-13T04:59:17.405 INFO:teuthology.orchestra.run.smithi001.stdout:4759 punch from 0x2bc1933 to 0x2bc8d5a, (0x7427 bytes) 2024-09-13T04:59:17.405 INFO:teuthology.orchestra.run.smithi001.stdout:4760 read 0xd78b635 thru 0xd79a0f0 (0xeabc bytes) 2024-09-13T04:59:17.419 INFO:teuthology.orchestra.run.smithi001.stdout:4761 trunc from 0xed72ff3 to 0x5d253c7 2024-09-13T04:59:17.449 INFO:teuthology.orchestra.run.smithi001.stdout:4763 punch from 0x4712b49 to 0x471a456, (0x790d bytes) 2024-09-13T04:59:17.449 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.450 INFO:teuthology.orchestra.run.smithi001.stdout:4766 write 0x4669bf2 thru 0x467252a (0x8939 bytes) 2024-09-13T04:59:17.477 INFO:teuthology.orchestra.run.smithi001.stdout:4767 writesame 0xda3d60d thru 0xda48715 data_size 0x1(0xb109 bytes) 2024-09-13T04:59:17.477 INFO:teuthology.orchestra.run.smithi001.stdout:4768 punch from 0x3967996 to 0x3973b65, (0xc1cf bytes) 2024-09-13T04:59:17.477 INFO:teuthology.orchestra.run.smithi001.stdout:4769 writesame 0x764bc61 thru 0x765ad3c data_size 0xa(0xf0dc bytes) 2024-09-13T04:59:17.489 INFO:teuthology.orchestra.run.smithi001.stdout:4770 read 0x277810 thru 0x2876b3 (0xfea4 bytes) 2024-09-13T04:59:17.490 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:17.487+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:17.491 INFO:teuthology.orchestra.run.smithi001.stdout:4771 read 0x142f712 thru 0x143499a (0x5289 bytes) 2024-09-13T04:59:17.514 INFO:teuthology.orchestra.run.smithi001.stdout:4772 trunc from 0xda48716 to 0x503c04a 2024-09-13T04:59:17.542 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.505786843999999, 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.543 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:17.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:17.547 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.505741207, 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:59:17.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:59:17.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.505693123, 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.552 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:59:17.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:17.557 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.505597041000001, 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:59:17.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.505545669999997, 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.562 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.563 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.566 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.505499616000002, 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:59:17.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:59:17.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.505449554999998, 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:59:17.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:59:17.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:59:17.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:59:17.575 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 61.388034365000003, 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.576 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:59:17.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:17.582 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:59:17.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:59:17.582 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.582 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296605982000003, 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:59:17.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296581637000003, 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.588 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:59:17.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.592 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:59:17.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:59:17.592 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.592 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.592 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.592 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296500283999997, 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:59:17.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:17.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:59:17.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:59:17.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.598 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296430753999999, 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:59:17.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.601 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296403478000002, 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.603 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:59:17.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:59:17.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296305033000003, 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:59:17.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:59:17.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296235465999999, 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.612 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:17.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:59:17.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.616 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296164992000001, 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:17.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:59:17.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296139306999997, 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:59:17.621 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:59:17.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296104118999999, 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:59:17.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296079603000003, 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.634 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 56.296010266000003, 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:17.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:17.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:17.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:17.639 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:17.639 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:17.640 INFO:teuthology.orchestra.run.smithi001.stdout:4774 writesame 0xeb209f9 thru 0xeb219a3 data_size 0x7(0xfab bytes) 2024-09-13T04:59:17.640 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.641 INFO:teuthology.orchestra.run.smithi001.stdout:4777 writesame 0x7d9c6c9 thru 0x7d9f261 data_size 0x1(0x2b99 bytes) 2024-09-13T04:59:17.641 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:17.641 INFO:teuthology.orchestra.run.smithi001.stdout:4780 read 0x8586706 thru 0x8596642 (0xff3d bytes) 2024-09-13T04:59:17.641 INFO:teuthology.orchestra.run.smithi001.stdout:4781 trunc from 0xeb219a4 to 0x24ce70a 2024-09-13T04:59:17.641 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:17.635+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:17.641 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:17.663 INFO:teuthology.orchestra.run.smithi001.stdout:4782 clone 30 order 23 su 4194304 sc 13 2024-09-13T04:59:17.742 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:17.738+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:17.769 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:17.769 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:17.769 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:17.783 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:17.843 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:17.843+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:17.943 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:17.938+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:17.946 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:17.946 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:17.946 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:17.946 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:17.946 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:17.960 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:18.048 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:18.046+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:18.085 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T04:59:18.104 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:59:18.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T04:59:18.135 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333821868000001, 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333787614000002, 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333756713, 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333704832000002, 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.157 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.158 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.158 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.158 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:18.158 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333673029000003, 2024-09-13T04:59:18.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:18.158 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:18.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:18.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:18.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:18.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:18.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:18.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:18.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333639075999997, 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.165 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:18.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:18.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:18.169 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.170 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.170 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.170 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.170 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.170 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:18.170 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333591339999998, 2024-09-13T04:59:18.171 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:18.171 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.171 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.171 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.171 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.172 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:18.173 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:18.174 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.175 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333551778, 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.176 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.177 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:18.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.178 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.178 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:18.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:18.178 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:18.181 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333504083999998, 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.183 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:18.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:18.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.186 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333441000999997, 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.187 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:18.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333404567000002, 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:18.192 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:18.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:18.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:18.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:18.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:18.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:18.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:18.198 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.198 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333370643999999, 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.199 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:18.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:18.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333302431, 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:18.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.206 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:18.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:18.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333268725000003, 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:18.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:18.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.333218533, 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:18.215 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.220 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.332912499999999, 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:18.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.225 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.332863297999999, 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:18.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.332815183999998, 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:18.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:18.234 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.332763706999998, 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:18.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 54.332728625999998, 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.239 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:18.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.244 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:18.246 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-13T04:59:18.247 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:18.243+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:18.247 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:18.346 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:18.343+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:18.424 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:18.424 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:18.424 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:18.424 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:18.438 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:18.447 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:18.443+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:18.478 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:18.548 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:18.543+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:18.560 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:18.560 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:18.561 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:18.561 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:18.561 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:18.575 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:18.629 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:18.629 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:18.646 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:18.646 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:18.648 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:18.646+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:18.663 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:18.663 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:18.681 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:18.681 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:18.684 INFO:teuthology.orchestra.run.smithi001.stdout:4783 write 0x4f8a8ab thru 0x4f9231a (0x7a70 bytes) 2024-09-13T04:59:18.691 INFO:teuthology.orchestra.run.smithi001.stdout:4786 punch from 0x4c4d6af to 0x4c53032, (0x5983 bytes) 2024-09-13T04:59:18.691 INFO:teuthology.orchestra.run.smithi001.stdout:4788 writesame 0x210dce8 thru 0x2116cd5 data_size 0x12(0x8fee bytes) 2024-09-13T04:59:18.696 INFO:teuthology.orchestra.run.smithi001.stdout:4789 read 0x3658ea9 thru 0x3662aee (0x9c46 bytes) 2024-09-13T04:59:18.702 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49885 3.b 3:d43de71f:::rbd_data.112a40bf6452.0000000000000003:f [sparse-read 0~1843200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:18.702 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 42.672300647, 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015118388, 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49885 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028150+0000", 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028157+0000", 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5510000000000002e-06 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028174+0000", 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6631000000000001e-05 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028183+0000", 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6100000000000006e-06 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028308+0000", 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012532600000000001 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028557+0000", 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024922200000000001 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.043270+0000", 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014712578 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.706 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49972 3.b 3:d2a26831:::rbd_data.112ae3a54590.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 42.521035241, 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021756569999999999, 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49972 2024-09-13T04:59:18.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179414+0000", 2024-09-13T04:59:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180536+0000", 2024-09-13T04:59:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001122381 2024-09-13T04:59:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180563+0000", 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6211999999999999e-05 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180582+0000", 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9630999999999999e-05 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180662+0000", 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9411000000000003e-05 2024-09-13T04:59:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180947+0000", 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000285166 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181294+0000", 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034697800000000001 2024-09-13T04:59:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200928+0000", 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019634766000000001 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201089+0000", 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016092299999999999 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201142+0000", 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2855000000000001e-05 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201173+0000", 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1096999999999997e-05 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49973 3.b 3:d63ae98e:::rbd_data.112ae3a54590.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 42.519841702000001, 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026409885000000001, 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49973 2024-09-13T04:59:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180608+0000", 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181851+0000", 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012428630000000001 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181894+0000", 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2710000000000003e-05 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181909+0000", 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5092999999999999e-05 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.186892+0000", 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049824229999999997 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187123+0000", 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023144999999999999 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187416+0000", 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029303400000000001 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200934+0000", 2024-09-13T04:59:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013517714 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206897+0000", 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059634880000000003 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206972+0000", 2024-09-13T04:59:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5165000000000003e-05 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207020+0000", 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7821000000000002e-05 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49975 3.b 3:d71a7fd7:::rbd_data.112ae3a54590.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1826816~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 42.518182942000003, 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025198755, 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49975 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182267+0000", 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183414+0000", 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011469270000000001 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183442+0000", 2024-09-13T04:59:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8189000000000001e-05 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183458+0000", 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6073999999999999e-05 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.195878+0000", 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01241922 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196039+0000", 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161058 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196279+0000", 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240676 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201295+0000", 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0050153999999999997 2024-09-13T04:59:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207382+0000", 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0060871650000000003 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207432+0000", 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9882000000000001e-05 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207468+0000", 2024-09-13T04:59:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6140000000000003e-05 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:49974 3.0 e48/24 3:03df0275:::rbd_data.112ae3a54590.000000000000000e:head v 48'299, mlcod=48'297)", 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.184595+0000", 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 42.515857502999999, 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010269866000000001, 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.184595+0000", 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.184595+0000", 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.184593+0000", 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.185341+0000", 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00074797399999999999 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.185364+0000", 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3280000000000001e-05 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.185375+0000", 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0253e-05 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.185429+0000", 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.452e-05 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.185461+0000", 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2119999999999997e-05 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.194810+0000", 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0093483720000000006 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.194864+0000", 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4839999999999997e-05 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79481 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.249117034000001, 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010622207, 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79481 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451331+0000", 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451350+0000", 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8448999999999999e-05 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451425+0000", 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4929e-05 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451448+0000", 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3618999999999999e-05 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451655+0000", 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020621600000000001 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451671+0000", 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6013000000000001e-05 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461479+0000", 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098082340000000007 2024-09-13T04:59:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461702+0000", 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022309900000000001 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461957+0000", 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000255171 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79525 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.248962827, 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010933693, 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79525 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451488+0000", 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451497+0000", 2024-09-13T04:59:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8359999999999998e-06 2024-09-13T04:59:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451527+0000", 2024-09-13T04:59:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0015999999999999e-05 2024-09-13T04:59:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451543+0000", 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6059999999999999e-05 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451797+0000", 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025463100000000001 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451810+0000", 2024-09-13T04:59:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2999999999999999e-05 2024-09-13T04:59:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462017+0000", 2024-09-13T04:59:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010207101999999999 2024-09-13T04:59:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462210+0000", 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192649 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462423+0000", 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021292899999999999 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79652 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455147+0000", 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.245304839999999, 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0096260019999999998, 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79652 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455147+0000", 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455147+0000", 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455146+0000", 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455153+0000", 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8630000000000002e-06 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455170+0000", 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6546999999999999e-05 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455182+0000", 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2726e-05 2024-09-13T04:59:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455799+0000", 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00061647700000000004 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455809+0000", 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0699999999999999e-05 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.464336+0000", 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0085265919999999995 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.464538+0000", 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020210200000000001 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.464773+0000", 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023506800000000001 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79695 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.245258741000001, 2024-09-13T04:59:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010472398000000001, 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79695 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455192+0000", 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455198+0000", 2024-09-13T04:59:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9399999999999999e-06 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455215+0000", 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.632e-05 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455229+0000", 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4146e-05 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455854+0000", 2024-09-13T04:59:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00062516999999999998 2024-09-13T04:59:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455860+0000", 2024-09-13T04:59:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7300000000000002e-06 2024-09-13T04:59:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.464832+0000", 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0089722559999999996 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465369+0000", 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00053748100000000003 2024-09-13T04:59:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465666+0000", 2024-09-13T04:59:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029624799999999999 2024-09-13T04:59:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79696 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.245211348000002, 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011064011, 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79696 2024-09-13T04:59:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455240+0000", 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455246+0000", 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5830000000000003e-06 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455264+0000", 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7450000000000001e-05 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455277+0000", 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3016e-05 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455886+0000", 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000609637 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455892+0000", 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5199999999999997e-06 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465781+0000", 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098892089999999995 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466021+0000", 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024036500000000001 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466305+0000", 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028331100000000001 2024-09-13T04:59:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79718 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.245163485999999, 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011634479, 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79718 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455288+0000", 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455293+0000", 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455310+0000", 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6393e-05 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455323+0000", 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3013000000000001e-05 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455918+0000", 2024-09-13T04:59:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059522399999999999 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455923+0000", 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4060000000000004e-06 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466385+0000", 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010462154 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466593+0000", 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020723900000000001 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466923+0000", 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00033036300000000002 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79737 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.245118160000001, 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011808300000000001, 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79737 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455333+0000", 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455339+0000", 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7570000000000001e-06 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455355+0000", 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.632e-05 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455368+0000", 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2588999999999999e-05 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455948+0000", 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058065099999999995 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455954+0000", 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466957+0000", 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011003361 2024-09-13T04:59:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467033+0000", 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5644999999999998e-05 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467142+0000", 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109527 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79757 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 393216~38443] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.245072848, 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013373497999999999, 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79757 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455378+0000", 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455384+0000", 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6200000000000004e-06 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455400+0000", 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5973000000000001e-05 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455412+0000", 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2689e-05 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455979+0000", 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00056651800000000001 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455984+0000", 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.49e-06 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467237+0000", 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011252376999999999 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467453+0000", 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000216338 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468753+0000", 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012995459999999999 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79776 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.245028002000002, 2024-09-13T04:59:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013932595000000001, 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79776 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455423+0000", 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455429+0000", 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455446+0000", 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6877000000000001e-05 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455459+0000", 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2866e-05 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456009+0000", 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000550838 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456015+0000", 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6060000000000002e-06 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468961+0000", 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012946301 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469181+0000", 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000220075 2024-09-13T04:59:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469357+0000", 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000175279 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:79796 3.0 e60/24 3:003e46ae:::rbd_data.112acc4fa3fd.0000000000000000:head v 60'366, mlcod=58'364)", 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.209695324999998, 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025297777, 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490755+0000", 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494145+0000", 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033899300000000002 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494187+0000", 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1950999999999999e-05 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494201+0000", 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4134000000000001e-05 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494267+0000", 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5815999999999995e-05 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494296+0000", 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8597000000000001e-05 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516017+0000", 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021721285 2024-09-13T04:59:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516054+0000", 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7389999999999999e-05 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79853 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 24.156715138999999, 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043236985999999998, 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79853 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543734+0000", 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548313+0000", 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045782449999999999 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548339+0000", 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6948999999999999e-05 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548356+0000", 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6160000000000001e-05 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548459+0000", 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000103453 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548718+0000", 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025891599999999999 2024-09-13T04:59:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.549442+0000", 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00072383699999999996 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586544+0000", 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037102028000000002 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586859+0000", 2024-09-13T04:59:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000315237 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586924+0000", 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5315999999999996e-05 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586974+0000", 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9481000000000003e-05 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81749 3.b 3:d52afe2d:::rbd_data.112a86dc1e25.000000000000000d:head [stat,delete] snapc 18=[18] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.62824805, 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010905319, 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81749 2024-09-13T04:59:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072201+0000", 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072215+0000", 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3859000000000001e-05 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072244+0000", 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8612999999999999e-05 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072265+0000", 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0893000000000001e-05 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.073443+0000", 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011780359999999999 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.073753+0000", 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031037700000000003 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.074309+0000", 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00055601200000000004 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.076960+0000", 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002650856 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.082880+0000", 2024-09-13T04:59:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005919901 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.083013+0000", 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000133168 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.083109+0000", 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6163999999999996e-05 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81735 3.8 e64/24 3:148b359f:::rbd_data.112a86dc1e25.0000000000000006:head v 64'139, mlcod=64'137)", 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.622731829999999, 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011631831, 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077718+0000", 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078104+0000", 2024-09-13T04:59:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000385852 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078144+0000", 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9858999999999999e-05 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078163+0000", 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8666999999999999e-05 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078282+0000", 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000119784 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078334+0000", 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1712000000000002e-05 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.089290+0000", 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010955515000000001 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.089352+0000", 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2389000000000002e-05 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81782 3.c e64/24 3:33317f5e:::rbd_data.112a86dc1e25.000000000000000b:head v 64'410, mlcod=63'408)", 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.605399933000001, 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012792019999999999, 2024-09-13T04:59:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095051+0000", 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095635+0000", 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058456600000000001 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095660+0000", 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4374000000000001e-05 2024-09-13T04:59:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095670+0000", 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0631999999999999e-05 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095743+0000", 2024-09-13T04:59:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3015000000000005e-05 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095775+0000", 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2092999999999999e-05 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107755+0000", 2024-09-13T04:59:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011979949 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107844+0000", 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8858999999999997e-05 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81874 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [sparse-read 0~2423869] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 20.432734368999999, 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015773423000000002, 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81874 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267715+0000", 2024-09-13T04:59:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267729+0000", 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4399999999999999e-05 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267761+0000", 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2112e-05 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267776+0000", 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4823e-05 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267903+0000", 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000127014 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.268135+0000", 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023146899999999999 2024-09-13T04:59:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.283491+0000", 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015356491 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:18.818 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:18.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:18.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:18.821 INFO:teuthology.orchestra.run.smithi001.stdout:4790 write 0x62f5fd4 thru 0x62f7c31 (0x1c5e bytes) 2024-09-13T04:59:18.821 INFO:teuthology.orchestra.run.smithi001.stdout:4791 write 0x82e4699 thru 0x82e70d8 (0x2a40 bytes) 2024-09-13T04:59:18.821 INFO:teuthology.orchestra.run.smithi001.stdout:4792 punch from 0x38f1049 to 0x38f450c, (0x34c3 bytes) 2024-09-13T04:59:18.821 INFO:teuthology.orchestra.run.smithi001.stdout:4793 trunc from 0x82e70d9 to 0x8398e1e 2024-09-13T04:59:18.822 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:18.814+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:18.822 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:18.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:18.823 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:18.823 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:18.823 INFO:teuthology.orchestra.run.smithi001.stdout:4795 writesame 0xcd09925 thru 0xcd1478e data_size 0x5f(0xae6a bytes) 2024-09-13T04:59:18.823 INFO:teuthology.orchestra.run.smithi001.stdout:4796 writesame 0xaf03a14 thru 0xaf0a641 data_size 0x2(0x6c2e bytes) 2024-09-13T04:59:18.824 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:18.824 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:18.824 INFO:teuthology.orchestra.run.smithi001.stdout:4797 trunc from 0xcd1478f to 0x31a7c02 2024-09-13T04:59:18.825 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:18.826 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:18.908 INFO:teuthology.orchestra.run.smithi001.stdout:4798 write 0xaf8e378 thru 0xaf8e846 (0x4cf bytes) 2024-09-13T04:59:18.913 INFO:teuthology.orchestra.run.smithi001.stdout:4799 punch from 0xa70b614 to 0xa70e1c4, (0x2bb0 bytes) 2024-09-13T04:59:18.920 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:18.915+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:18.931 INFO:teuthology.orchestra.run.smithi001.stdout:4800 trunc from 0xaf8e847 to 0x798196 2024-09-13T04:59:18.973 INFO:teuthology.orchestra.run.smithi001.stdout:4801 trunc from 0x798196 to 0xeb5b51 2024-09-13T04:59:19.003 INFO:teuthology.orchestra.run.smithi001.stdout:4803 write 0x8a19ef0 thru 0x8a2007d (0x618e bytes) 2024-09-13T04:59:19.018 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:19.018 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:19.022 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:19.019+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:19.022 INFO:teuthology.orchestra.run.smithi001.stdout:4805 write 0x9fbb26a thru 0x9fc2680 (0x7417 bytes) 2024-09-13T04:59:19.028 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:19.031 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:19.040 INFO:teuthology.orchestra.run.smithi001.stdout:4807 writesame 0xa006166 thru 0xa00f0cc data_size 0x3(0x8f67 bytes) 2024-09-13T04:59:19.041 INFO:teuthology.orchestra.run.smithi001.stdout:4808 write 0x7c121e1 thru 0x7c1f3e6 (0xd206 bytes) 2024-09-13T04:59:19.053 INFO:teuthology.orchestra.run.smithi001.stdout:4809 write 0x30df957 thru 0x30e72cd (0x7977 bytes) 2024-09-13T04:59:19.058 INFO:teuthology.orchestra.run.smithi001.stdout:4810 read 0x73f515c thru 0x73f8acf (0x3974 bytes) 2024-09-13T04:59:19.072 INFO:teuthology.orchestra.run.smithi001.stdout:4811 write 0xbc88166 thru 0xbc8b7a5 (0x3640 bytes) 2024-09-13T04:59:19.075 INFO:teuthology.orchestra.run.smithi001.stdout:4812 read 0x63dc6b3 thru 0x63eab22 (0xe470 bytes) 2024-09-13T04:59:19.075 INFO:teuthology.orchestra.run.smithi001.stdout:4813 write 0x7b62cd1 thru 0x7b69b1c (0x6e4c bytes) 2024-09-13T04:59:19.085 INFO:teuthology.orchestra.run.smithi001.stdout:4814 trunc from 0xbc8b7a6 to 0xd807ddd 2024-09-13T04:59:19.098 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:59:19.098 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T04:59:19.111 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:19.111 INFO:teuthology.orchestra.run.smithi001.stdout:4816 read 0x26a34f8 thru 0x26b14be (0xdfc7 bytes) 2024-09-13T04:59:19.111 INFO:teuthology.orchestra.run.smithi001.stdout:4817 read 0xcfed641 thru 0xcffc30a (0xecca bytes) 2024-09-13T04:59:19.111 INFO:teuthology.orchestra.run.smithi001.stdout:4818 clone 31 order 20 su 524288 sc 15 2024-09-13T04:59:19.122 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T04:59:19.124 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:19.122+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:19.169 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:19.169 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:19.183 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:19.225 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:19.222+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:19.326 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:19.323+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:19.350 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:19.351 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:19.364 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:19.449 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:19.447+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:19.523 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:19.523 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:19.528 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:19.523+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:19.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:19.628 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:19.623+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:19.692 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:19.692 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:19.706 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:19.750 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:19.747+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:19.830 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:19.826+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:19.877 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:19.877 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:19.893 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:19.931 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:19.926+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:20.017 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:20.018 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:20.031 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:20.027+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:20.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:20.106 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T04:59:20.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T04:59:20.174 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:20.171+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:20.209 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:20.209 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:20.209 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone30 from 0xd807ddd (overlap 0x798196) to 0x6d5b07 2024-09-13T04:59:20.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:20.274 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:20.271+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:20.303 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-13T04:59:20.333 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:20.331+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:20.379 INFO:teuthology.orchestra.run.smithi001.stdout:4819 read 0xc755b77 thru 0xc75e5b7 (0x8a41 bytes) 2024-09-13T04:59:20.379 INFO:teuthology.orchestra.run.smithi001.stdout:4821 read 0x7c17eb6 thru 0x7c24489 (0xc5d4 bytes) 2024-09-13T04:59:20.382 INFO:teuthology.orchestra.run.smithi001.stdout:4824 read 0x71805b2 thru 0x718696f (0x63be bytes) 2024-09-13T04:59:20.383 INFO:teuthology.orchestra.run.smithi001.stdout:4825 read 0x97a25be thru 0x97a3a18 (0x145b bytes) 2024-09-13T04:59:20.384 INFO:teuthology.orchestra.run.smithi001.stdout:4826 read 0x86ac06f thru 0x86b0e3b (0x4dcd bytes) 2024-09-13T04:59:20.399 INFO:teuthology.orchestra.run.smithi001.stdout:4827 trunc from 0xd807ddd to 0x328311a 2024-09-13T04:59:20.400 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:20.400 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:20.417 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:20.435 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:20.430+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:20.452 INFO:teuthology.orchestra.run.smithi001.stdout:4828 trunc from 0x328311a to 0x2aba011 2024-09-13T04:59:20.490 INFO:teuthology.orchestra.run.smithi001.stdout:4829 punch from 0x2321516 to 0x232dfff, (0xcae9 bytes) 2024-09-13T04:59:20.490 INFO:teuthology.orchestra.run.smithi001.stdout:4830 punch from 0x1beba64 to 0x1bf4650, (0x8bec bytes) 2024-09-13T04:59:20.503 INFO:teuthology.orchestra.run.smithi001.stdout:4831 writesame 0x4922bcb thru 0x4932263 data_size 0xb(0xf699 bytes) 2024-09-13T04:59:20.503 INFO:teuthology.orchestra.run.smithi001.stdout:4833 read 0x35f9ce0 thru 0x3607926 (0xdc47 bytes) 2024-09-13T04:59:20.517 INFO:teuthology.orchestra.run.smithi001.stdout:4834 write 0x71fe918 thru 0x7207f50 (0x9639 bytes) 2024-09-13T04:59:20.532 INFO:teuthology.orchestra.run.smithi001.stdout:4835 trunc from 0x7207f51 to 0x25bf8f6 2024-09-13T04:59:20.535 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:20.530+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:20.583 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:20.583 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:20.584 INFO:teuthology.orchestra.run.smithi001.stdout:4836 punch from 0x9e52fd to 0x9f3dab, (0xeaae bytes) 2024-09-13T04:59:20.584 INFO:teuthology.orchestra.run.smithi001.stdout:4839 read 0x18999c thru 0x18f62f (0x5c94 bytes) 2024-09-13T04:59:20.584 INFO:teuthology.orchestra.run.smithi001.stdout:4840 read 0x25026e9 thru 0x25114fd (0xee15 bytes) 2024-09-13T04:59:20.596 INFO:teuthology.orchestra.run.smithi001.stdout:4843 trunc from 0x25bf8f6 to 0xa4532e2 2024-09-13T04:59:20.605 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:20.618 INFO:teuthology.orchestra.run.smithi001.stdout:4844 trunc from 0xa4532e2 to 0x57b8c6d 2024-09-13T04:59:20.630 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:20.631 INFO:teuthology.orchestra.run.smithi001.stdout:4848 trunc from 0x57b8c6d to 0x5fce7b6 2024-09-13T04:59:20.637 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:20.634+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:20.662 INFO:teuthology.orchestra.run.smithi001.stdout:4850 write 0x822b8a6 thru 0x8231954 (0x60af bytes) 2024-09-13T04:59:20.669 INFO:teuthology.orchestra.run.smithi001.stdout:4852 writesame 0xeba7b6 thru 0xebaea1 data_size 0x2(0x6ec bytes) 2024-09-13T04:59:20.696 INFO:teuthology.orchestra.run.smithi001.stdout:4853 write 0xb5d02c4 thru 0xb5d8fed (0x8d2a bytes) 2024-09-13T04:59:20.704 INFO:teuthology.orchestra.run.smithi001.stdout:4854 punch from 0x5aef81d to 0x5afdf38, (0xe71b bytes) 2024-09-13T04:59:20.704 INFO:teuthology.orchestra.run.smithi001.stdout:4857 read 0x71b3495 thru 0x71be04e (0xabba bytes) 2024-09-13T04:59:20.704 INFO:teuthology.orchestra.run.smithi001.stdout:4858 read 0x56fd66e thru 0x5702402 (0x4d95 bytes) 2024-09-13T04:59:20.704 INFO:teuthology.orchestra.run.smithi001.stdout:4859 write 0x6a87e01 thru 0x6a8d129 (0x5329 bytes) 2024-09-13T04:59:20.731 INFO:teuthology.orchestra.run.smithi001.stdout:4860 write 0xddde9bb thru 0xdde8b94 (0xa1da bytes) 2024-09-13T04:59:20.738 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:20.735+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:20.739 INFO:teuthology.orchestra.run.smithi001.stdout:4862 punch from 0xa63878 to 0xa68b7b, (0x5303 bytes) 2024-09-13T04:59:20.739 INFO:teuthology.orchestra.run.smithi001.stdout:4863 write 0x47c1666 thru 0x47c5e0d (0x47a8 bytes) 2024-09-13T04:59:20.747 INFO:teuthology.orchestra.run.smithi001.stdout:4866 punch from 0x231f734 to 0x2328d9b, (0x9667 bytes) 2024-09-13T04:59:20.761 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:20.761 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:20.763 INFO:teuthology.orchestra.run.smithi001.stdout:4867 write 0xe9644c3 thru 0xe965703 (0x1241 bytes) 2024-09-13T04:59:20.778 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:20.801 INFO:teuthology.orchestra.run.smithi001.stdout:4868 trunc from 0xe965704 to 0xdc7611 2024-09-13T04:59:20.826 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:20.846 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:20.843+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:20.851 INFO:teuthology.orchestra.run.smithi001.stdout:4869 write 0x228f155 thru 0x229c44f (0xd2fb bytes) 2024-09-13T04:59:20.859 INFO:teuthology.orchestra.run.smithi001.stdout:4871 read 0x18881f3 thru 0x189114e (0x8f5c bytes) 2024-09-13T04:59:20.885 INFO:teuthology.orchestra.run.smithi001.stdout:4872 write 0xba56008 thru 0xba624e5 (0xc4de bytes) 2024-09-13T04:59:20.891 INFO:teuthology.orchestra.run.smithi001.stdout:4873 write 0x3cb5279 thru 0x3cb74d7 (0x225f bytes) 2024-09-13T04:59:20.897 INFO:teuthology.orchestra.run.smithi001.stdout:4874 read 0xac97395 thru 0xac9973b (0x23a7 bytes) 2024-09-13T04:59:20.897 INFO:teuthology.orchestra.run.smithi001.stdout:4876 read 0xf2b476 thru 0xf30342 (0x4ecd bytes) 2024-09-13T04:59:20.897 INFO:teuthology.orchestra.run.smithi001.stdout:4877 write 0x4d3cd68 thru 0x4d3e5da (0x1873 bytes) 2024-09-13T04:59:20.902 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:20.902 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:20.903 INFO:teuthology.orchestra.run.smithi001.stdout:4881 trunc from 0xba624e6 to 0xb50ee1c 2024-09-13T04:59:20.923 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:20.927 INFO:teuthology.orchestra.run.smithi001.stdout:4885 trunc from 0xb50ee1c to 0xd39b88f 2024-09-13T04:59:20.932 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:20.932 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:20.939 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:20.935+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:20.941 INFO:teuthology.orchestra.run.smithi001.stdout:4886 read 0xcf98b30 thru 0xcfa4ce6 (0xc1b7 bytes) 2024-09-13T04:59:20.941 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:20.941 INFO:teuthology.orchestra.run.smithi001.stdout:4888 read 0x69c113e thru 0x69c5b2c (0x49ef bytes) 2024-09-13T04:59:20.941 INFO:teuthology.orchestra.run.smithi001.stdout:4889 punch from 0x60d0577 to 0x60d36f5, (0x317e bytes) 2024-09-13T04:59:20.941 INFO:teuthology.orchestra.run.smithi001.stdout:4890 read 0xbc90fab thru 0xbc96987 (0x59dd bytes) 2024-09-13T04:59:20.941 INFO:teuthology.orchestra.run.smithi001.stdout:4892 read 0x2aff429 thru 0x2b056b3 (0x628b bytes) 2024-09-13T04:59:20.941 INFO:teuthology.orchestra.run.smithi001.stdout:4893 write 0x468cfcf thru 0x469cb98 (0xfbca bytes) 2024-09-13T04:59:20.951 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:20.952 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:20.952 INFO:teuthology.orchestra.run.smithi001.stdout:4895 read 0x485322f thru 0x485c954 (0x9726 bytes) 2024-09-13T04:59:20.953 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:20.953 INFO:teuthology.orchestra.run.smithi001.stdout:4897 read 0x74e84f7 thru 0x74efe84 (0x798e bytes) 2024-09-13T04:59:20.953 INFO:teuthology.orchestra.run.smithi001.stdout:4898 read 0x83bbd1f thru 0x83bf40f (0x36f1 bytes) 2024-09-13T04:59:20.953 INFO:teuthology.orchestra.run.smithi001.stdout:4899 read 0x15b8c89 thru 0x15bd868 (0x4be0 bytes) 2024-09-13T04:59:20.953 INFO:teuthology.orchestra.run.smithi001.stdout:4900 read 0x28d9058 thru 0x28dac03 (0x1bac bytes) 2024-09-13T04:59:20.968 INFO:teuthology.orchestra.run.smithi001.stdout:4901 trunc from 0xd39b88f to 0x274eaee 2024-09-13T04:59:21.008 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:21.008 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:21.009 INFO:teuthology.orchestra.run.smithi001.stdout:4902 writesame 0xb5e4860 thru 0xb5ef88d data_size 0x2(0xb02e bytes) 2024-09-13T04:59:21.009 INFO:teuthology.orchestra.run.smithi001.stdout:4903 punch from 0x872344a to 0x8726f6a, (0x3b20 bytes) 2024-09-13T04:59:21.009 INFO:teuthology.orchestra.run.smithi001.stdout:4904 write 0x91e05fb thru 0x91eca17 (0xc41d bytes) 2024-09-13T04:59:21.016 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.016 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.026 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:21.026 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:21.029 INFO:teuthology.orchestra.run.smithi001.stdout:4910 write 0xe63cef9 thru 0xe63ffea (0x30f2 bytes) 2024-09-13T04:59:21.034 INFO:teuthology.orchestra.run.smithi001.stdout:4911 read 0x581b730 thru 0x5822ef5 (0x77c6 bytes) 2024-09-13T04:59:21.034 INFO:teuthology.orchestra.run.smithi001.stdout:4912 read 0x2946b88 thru 0x2949741 (0x2bba bytes) 2024-09-13T04:59:21.034 INFO:teuthology.orchestra.run.smithi001.stdout:4913 read 0x4bc32ec thru 0x4bd1f71 (0xec86 bytes) 2024-09-13T04:59:21.034 INFO:teuthology.orchestra.run.smithi001.stdout:4914 punch from 0x37d63be to 0x37dd34d, (0x6f8f bytes) 2024-09-13T04:59:21.034 INFO:teuthology.orchestra.run.smithi001.stdout:4915 read 0xde72456 thru 0xde7f047 (0xcbf2 bytes) 2024-09-13T04:59:21.035 INFO:teuthology.orchestra.run.smithi001.stdout:4916 write 0xc8972c9 thru 0xc8996cf (0x2407 bytes) 2024-09-13T04:59:21.038 INFO:teuthology.orchestra.run.smithi001.stdout:4917 punch from 0x2f246ce to 0x2f24af9, (0x42b bytes) 2024-09-13T04:59:21.038 INFO:teuthology.orchestra.run.smithi001.stdout:4918 read 0x92bf9c1 thru 0x92c9507 (0x9b47 bytes) 2024-09-13T04:59:21.038 INFO:teuthology.orchestra.run.smithi001.stdout:4919 write 0x448e4b8 thru 0x449aa02 (0xc54b bytes) 2024-09-13T04:59:21.041 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:21.038+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:21.044 INFO:teuthology.orchestra.run.smithi001.stdout:4920 writesame 0x93c9f72 thru 0x93d6529 data_size 0x26(0xc5b8 bytes) 2024-09-13T04:59:21.045 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:21.045 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:21.050 INFO:teuthology.orchestra.run.smithi001.stdout:4923 write 0x95cb98c thru 0x95d7b11 (0xc186 bytes) 2024-09-13T04:59:21.057 INFO:teuthology.orchestra.run.smithi001.stdout:4924 read 0x5e70fb2 thru 0x5e78b8d (0x7bdc bytes) 2024-09-13T04:59:21.061 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:21.061 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:21.068 INFO:teuthology.orchestra.run.smithi001.stdout:4925 write 0xe772b86 thru 0xe78062e (0xdaa9 bytes) 2024-09-13T04:59:21.076 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.076 INFO:teuthology.orchestra.run.smithi001.stdout:4927 read 0x98bb308 thru 0x98c074d (0x5446 bytes) 2024-09-13T04:59:21.076 INFO:teuthology.orchestra.run.smithi001.stdout:4930 read 0x2b86c68 thru 0x2b903bc (0x9755 bytes) 2024-09-13T04:59:21.076 INFO:teuthology.orchestra.run.smithi001.stdout:4932 read 0x8a8041d thru 0x8a8a1af (0x9d93 bytes) 2024-09-13T04:59:21.077 INFO:teuthology.orchestra.run.smithi001.stdout:4934 punch from 0x23d938c to 0x23dcded, (0x3a61 bytes) 2024-09-13T04:59:21.077 INFO:teuthology.orchestra.run.smithi001.stdout:4935 read 0xe0b35b7 thru 0xe0bf06b (0xbab5 bytes) 2024-09-13T04:59:21.077 INFO:teuthology.orchestra.run.smithi001.stdout:4936 trunc from 0xe78062f to 0xdd17711 2024-09-13T04:59:21.085 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:21.096 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:21.097 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:21.097 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:21.097 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:21.098 INFO:teuthology.orchestra.run.smithi001.stdout:4937 read 0x58b5d73 thru 0x58bbc7e (0x5f0c bytes) 2024-09-13T04:59:21.098 INFO:teuthology.orchestra.run.smithi001.stdout:4938 write 0x8abb9dd thru 0x8ac1cfc (0x6320 bytes) 2024-09-13T04:59:21.108 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T04:59:21.108 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.108 INFO:teuthology.orchestra.run.smithi001.stdout:4941 read 0xcc09632 thru 0xcc0f7e6 (0x61b5 bytes) 2024-09-13T04:59:21.108 INFO:teuthology.orchestra.run.smithi001.stdout:4942 punch from 0x74d5abb to 0x74ddf92, (0x84d7 bytes) 2024-09-13T04:59:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:4945 read 0xc92b828 thru 0xc938109 (0xc8e2 bytes) 2024-09-13T04:59:21.109 INFO:teuthology.orchestra.run.smithi001.stdout:4946 writesame 0x273a519 thru 0x2741fb6 data_size 0xa(0x7a9e bytes) 2024-09-13T04:59:21.110 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:21.110 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:21.113 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:21.118 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.118 INFO:teuthology.orchestra.run.smithi001.stdout:4948 read 0x3ac4e8b thru 0x3aca1f3 (0x5369 bytes) 2024-09-13T04:59:21.118 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.118 INFO:teuthology.orchestra.run.smithi001.stdout:4950 writesame 0x1a428b7 thru 0x1a45e60 data_size 0x1(0x35aa bytes) 2024-09-13T04:59:21.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:59:21.129 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T04:59:21.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:59:21.132 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:21.133 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:21.134 INFO:teuthology.orchestra.run.smithi001.stdout:4951 trunc from 0xdd17711 to 0xd9d9421 2024-09-13T04:59:21.141 INFO:teuthology.orchestra.run.smithi001.stdout:4953 read 0x1587e00 thru 0x1590887 (0x8a88 bytes) 2024-09-13T04:59:21.142 INFO:teuthology.orchestra.run.smithi001.stdout:4956 write 0x9fed83c thru 0x9ff86ec (0xaeb1 bytes) 2024-09-13T04:59:21.150 INFO:teuthology.orchestra.run.smithi001.stdout:4958 write 0xbbf00e6 thru 0xbbfae89 (0xada4 bytes) 2024-09-13T04:59:21.155 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:21.156 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:21.156 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:21.154+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:21.160 INFO:teuthology.orchestra.run.smithi001.stdout:4959 write 0x40e2425 thru 0x40ec758 (0xa334 bytes) 2024-09-13T04:59:21.168 INFO:teuthology.orchestra.run.smithi001.stdout:4960 read 0xbbe32f9 thru 0xbbe4112 (0xe1a bytes) 2024-09-13T04:59:21.168 INFO:teuthology.orchestra.run.smithi001.stdout:4964 read 0x61fc87c thru 0x61fec4c (0x23d1 bytes) 2024-09-13T04:59:21.170 INFO:teuthology.orchestra.run.smithi001.stdout:4966 trunc from 0xd9d9421 to 0xc70571f 2024-09-13T04:59:21.185 INFO:teuthology.orchestra.run.smithi001.stdout:4967 read 0x4deadfb thru 0x4df172a (0x6930 bytes) 2024-09-13T04:59:21.185 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.185 INFO:teuthology.orchestra.run.smithi001.stdout:4969 write 0xacef74b thru 0xacfdca9 (0xe55f bytes) 2024-09-13T04:59:21.192 INFO:teuthology.orchestra.run.smithi001.stdout:4970 write 0x3a09a73 thru 0x3a0f8bb (0x5e49 bytes) 2024-09-13T04:59:21.197 INFO:teuthology.orchestra.run.smithi001.stdout:4971 writesame 0xb3f645 thru 0xb44538 data_size 0x3e(0x4ef4 bytes) 2024-09-13T04:59:21.209 INFO:teuthology.orchestra.run.smithi001.stdout:4972 punch from 0x3378f9b to 0x3385d78, (0xcddd bytes) 2024-09-13T04:59:21.223 INFO:teuthology.orchestra.run.smithi001.stdout:4973 trunc from 0xc70571f to 0x3a1b90d 2024-09-13T04:59:21.242 INFO:teuthology.orchestra.run.smithi001.stdout:4975 trunc from 0x3a1b90d to 0x28434e3 2024-09-13T04:59:21.243 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:21.239+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:21.257 INFO:teuthology.orchestra.run.smithi001.stdout:4976 punch from 0x1f5fed2 to 0x1f603a5, (0x4d3 bytes) 2024-09-13T04:59:21.272 INFO:teuthology.orchestra.run.smithi001.stdout:4977 write 0x846c1a4 thru 0x847a5fe (0xe45b bytes) 2024-09-13T04:59:21.277 INFO:teuthology.orchestra.run.smithi001.stdout:4978 read 0x438aa4e thru 0x4391c7a (0x722d bytes) 2024-09-13T04:59:21.277 INFO:teuthology.orchestra.run.smithi001.stdout:4980 read 0x7681c72 thru 0x7686d6b (0x50fa bytes) 2024-09-13T04:59:21.277 INFO:teuthology.orchestra.run.smithi001.stdout:4981 write 0x59fb638 thru 0x5a01e2f (0x67f8 bytes) 2024-09-13T04:59:21.281 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:21.281 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:21.282 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:21.282 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:21.282 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:21.286 INFO:teuthology.orchestra.run.smithi001.stdout:4983 trunc from 0x847a5ff to 0x9e8a722 2024-09-13T04:59:21.296 INFO:teuthology.orchestra.run.smithi001.stdout:4984 writesame 0x4297474 thru 0x429ecc0 data_size 0x17(0x784d bytes) 2024-09-13T04:59:21.298 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:21.309 INFO:teuthology.orchestra.run.smithi001.stdout:4986 trunc from 0x9e8a722 to 0xe86fc48 2024-09-13T04:59:21.315 INFO:teuthology.orchestra.run.smithi001.stdout:4990 write 0x9b12598 thru 0x9b1c9e3 (0xa44c bytes) 2024-09-13T04:59:21.321 INFO:teuthology.orchestra.run.smithi001.stdout:4993 writesame 0x18fc9ca thru 0x18feb3b data_size 0x3(0x2172 bytes) 2024-09-13T04:59:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:4994 write 0x7e9255b thru 0x7e9acfc (0x87a2 bytes) 2024-09-13T04:59:21.333 INFO:teuthology.orchestra.run.smithi001.stdout:4996 punch from 0x76759cb to 0x767a175, (0x47aa bytes) 2024-09-13T04:59:21.333 INFO:teuthology.orchestra.run.smithi001.stdout:4997 write 0x254b43b thru 0x2556394 (0xaf5a bytes) 2024-09-13T04:59:21.344 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:21.339+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:21.349 INFO:teuthology.orchestra.run.smithi001.stdout:4998 trunc from 0xe86fc48 to 0x97bcdc3 2024-09-13T04:59:21.364 INFO:teuthology.orchestra.run.smithi001.stdout:4999 trunc from 0x97bcdc3 to 0xa795b80 2024-09-13T04:59:21.371 INFO:teuthology.orchestra.run.smithi001.stdout:5000 write 0xa086987 thru 0xa088d0a (0x2384 bytes) 2024-09-13T04:59:21.375 INFO:teuthology.orchestra.run.smithi001.stdout:5001 trunc from 0xa795b80 to 0xa518ad5 2024-09-13T04:59:21.381 INFO:teuthology.orchestra.run.smithi001.stdout:5002 punch from 0xa4f3e94 to 0xa4fefbb, (0xb127 bytes) 2024-09-13T04:59:21.386 INFO:teuthology.orchestra.run.smithi001.stdout:5005 trunc from 0xa518ad5 to 0xcc31f88 2024-09-13T04:59:21.393 INFO:teuthology.orchestra.run.smithi001.stdout:5006 write 0x6ef3540 thru 0x6ef7eaa (0x496b bytes) 2024-09-13T04:59:21.397 INFO:teuthology.orchestra.run.smithi001.stdout:5007 writesame 0x3a34cf thru 0x3aa530 data_size 0x69(0x7062 bytes) 2024-09-13T04:59:21.401 INFO:teuthology.orchestra.run.smithi001.stdout:5008 punch from 0x3838e3e to 0x3841884, (0x8a46 bytes) 2024-09-13T04:59:21.401 INFO:teuthology.orchestra.run.smithi001.stdout:5010 read 0x5e7473e thru 0x5e753db (0xc9e bytes) 2024-09-13T04:59:21.401 INFO:teuthology.orchestra.run.smithi001.stdout:5011 read 0xb842ba3 thru 0xb843b58 (0xfb6 bytes) 2024-09-13T04:59:21.401 INFO:teuthology.orchestra.run.smithi001.stdout:5012 writesame 0x19aedd6 thru 0x19bb6db data_size 0x1b(0xc906 bytes) 2024-09-13T04:59:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:5013 read 0x5f39d thru 0x69612 (0xa276 bytes) 2024-09-13T04:59:21.409 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.409 INFO:teuthology.orchestra.run.smithi001.stdout:5018 write 0x7fbf8c2 thru 0x7fc4b10 (0x524f bytes) 2024-09-13T04:59:21.414 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.422 INFO:teuthology.orchestra.run.smithi001.stdout:5022 write 0xd037654 thru 0xd039a84 (0x2431 bytes) 2024-09-13T04:59:21.428 INFO:teuthology.orchestra.run.smithi001.stdout:5023 write 0x91773f thru 0x91e4cb (0x6d8d bytes) 2024-09-13T04:59:21.431 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:21.431 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.570633267000002, 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.436 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.570606355999999, 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T04:59:21.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.566860720000001, 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T04:59:21.441 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T04:59:21.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.563317394999999, 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T04:59:21.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T04:59:21.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.563277766999999, 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T04:59:21.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.563254106999999, 2024-09-13T04:59:21.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T04:59:21.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T04:59:21.459 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.563166557999999, 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.460 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.464 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.563113902000001, 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T04:59:21.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.559188011, 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T04:59:21.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.559162386000001, 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T04:59:21.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.478 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.559128992000002, 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:21.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T04:59:21.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.554339063, 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T04:59:21.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.550944434000002, 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T04:59:21.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.550871758, 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.492 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T04:59:21.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.547129396999999, 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T04:59:21.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.546956406, 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.501 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T04:59:21.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T04:59:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:21.505 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.543113807000001, 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T04:59:21.509 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.503265829, 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T04:59:21.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.514 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.498936607000001, 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T04:59:21.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T04:59:21.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:21.519 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 27.491207607, 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:21.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:59:21.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T04:59:21.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T04:59:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:21.524 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:21.524 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:21.525 INFO:teuthology.orchestra.run.smithi001.stdout:5027 trunc from 0xd039a85 to 0x526bae4 2024-09-13T04:59:21.527 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:59:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:59:21.528 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:21.528 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:21.522+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:21.529 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.529 INFO:teuthology.orchestra.run.smithi001.stdout:5029 read 0x5073a93 thru 0x5082277 (0xe7e5 bytes) 2024-09-13T04:59:21.530 INFO:teuthology.orchestra.run.smithi001.stdout:5030 punch from 0x2e43a13 to 0x2e47bc3, (0x41b0 bytes) 2024-09-13T04:59:21.530 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.530 INFO:teuthology.orchestra.run.smithi001.stdout:5033 punch from 0xc614fa to 0xc6f0fc, (0xdc02 bytes) 2024-09-13T04:59:21.530 INFO:teuthology.orchestra.run.smithi001.stdout:5035 read 0x50aa1cc thru 0x50ac10d (0x1f42 bytes) 2024-09-13T04:59:21.530 INFO:teuthology.orchestra.run.smithi001.stdout:5036 punch from 0x1ab3b77 to 0x1abcf89, (0x9412 bytes) 2024-09-13T04:59:21.530 INFO:teuthology.orchestra.run.smithi001.stdout:5037 trunc from 0x526bae4 to 0x673663c 2024-09-13T04:59:21.530 INFO:teuthology.orchestra.run.smithi001.stdout:5038 read 0x668dbf2 thru 0x668e28f (0x69e bytes) 2024-09-13T04:59:21.531 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-13T04:59:21.104505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1007384064,"num_objects":1773,"num_object_clones":417,"num_object_copies":3546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1773,"num_whiteouts":828,"num_read":120648,"num_read_kb":565441,"num_write":14267,"num_write_kb":273059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11499,"ondisk_log_size":11499,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":319472,"kb_used_data":58260,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754655248,"statfs":{"total":773094113280,"available":772766973952,"internally_reserved":0,"allocated":59658240,"data_stored":417651221,"data_compressed":16282157,"data_compressed_allocated":40726528,"data_compressed_original":406071444,"omap_allocated":12743,"internal_metadata":267374137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19683856,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":3667,"num_read_kb":21130,"num_write":857,"num_write_kb":7669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035219"},"pg_stats":[{"pgid":"3.1f","version":"77'389","reported_seq":2607,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T04:59:15.351646+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:59:15.351646+0000","last_peered":"2024-09-13T04:59:15.351646+0000","last_clean":"2024-09-13T04:59:15.351646+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:59:15.351646+0000","last_undegraded":"2024-09-13T04:59:15.351646+0000","last_fullsized":"2024-09-13T04:59:15.351646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":51629431,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":3731,"num_read_kb":15540,"num_write":438,"num_write_kb":8468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"76'231","reported_seq":1709,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T04:59:15.073332+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:59:15.073332+0000","last_peered":"2024-09-13T04:59:15.073332+0000","last_clean":"2024-09-13T04:59:15.073332+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:59:15.073332+0000","last_undegraded":"2024-09-13T04:59:15.073332+0000","last_fullsized":"2024-09-13T04:59:15.073332+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":17572659,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":2351,"num_read_kb":14458,"num_write":218,"num_write_kb":5747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"79'417","reported_seq":1084,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.491355+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:59:17.491355+0000","last_peered":"2024-09-13T04:59:17.491355+0000","last_clean":"2024-09-13T04:59:17.491355+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:59:17.491355+0000","last_undegraded":"2024-09-13T04:59:17.491355+0000","last_fullsized":"2024-09-13T04:59:17.491355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":48762137,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":31,"num_read":1743,"num_read_kb":13131,"num_write":576,"num_write_kb":8676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"76'316","reported_seq":984,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.086848+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:59:16.086848+0000","last_peered":"2024-09-13T04:59:16.086848+0000","last_clean":"2024-09-13T04:59:16.086848+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:59:16.086848+0000","last_undegraded":"2024-09-13T04:59:16.086848+0000","last_fullsized":"2024-09-13T04:59:16.086848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":30471990,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":1231,"num_read_kb":6683,"num_write":364,"num_write_kb":6514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'341","reported_seq":4603,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.092031+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:59:16.092031+0000","last_peered":"2024-09-13T04:59:16.092031+0000","last_clean":"2024-09-13T04:59:16.092031+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:59:16.092031+0000","last_undegraded":"2024-09-13T04:59:16.092031+0000","last_fullsized":"2024-09-13T04:59:16.092031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":25770403,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":5294,"num_read_kb":15720,"num_write":393,"num_write_kb":5849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"79'180","reported_seq":2061,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.576386+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:59:17.576386+0000","last_peered":"2024-09-13T04:59:17.576386+0000","last_clean":"2024-09-13T04:59:17.576386+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:59:17.576386+0000","last_undegraded":"2024-09-13T04:59:17.576386+0000","last_fullsized":"2024-09-13T04:59:17.576386+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":24036588,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":1961,"num_read_kb":10026,"num_write":185,"num_write_kb":7409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"78'335","reported_seq":17576,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.495531+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:59:16.495531+0000","last_peered":"2024-09-13T04:59:16.495531+0000","last_clean":"2024-09-13T04:59:16.495531+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:59:16.495531+0000","last_undegraded":"2024-09-13T04:59:16.495531+0000","last_fullsized":"2024-09-13T04:59:16.495531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":46179149,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":18227,"num_read_kb":95531,"num_write":398,"num_write_kb":18454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"72'333","reported_seq":2469,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T04:59:15.073608+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:59:15.073608+0000","last_peered":"2024-09-13T04:59:15.073608+0000","last_clean":"2024-09-13T04:59:15.073608+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:59:15.073608+0000","last_undegraded":"2024-09-13T04:59:15.073608+0000","last_fullsized":"2024-09-13T04:59:15.073608+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":26896124,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":31,"num_read":2905,"num_read_kb":8958,"num_write":422,"num_write_kb":5185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"78'211","reported_seq":2899,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.543883+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:59:16.543883+0000","last_peered":"2024-09-13T04:59:16.543883+0000","last_clean":"2024-09-13T04:59:16.543883+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:59:16.543883+0000","last_undegraded":"2024-09-13T04:59:16.543883+0000","last_fullsized":"2024-09-13T04:59:16.543883+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":27984346,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":2682,"num_read_kb":21102,"num_write":224,"num_write_kb":13793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"75'381","reported_seq":1685,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.062347+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:59:16.062347+0000","last_peered":"2024-09-13T04:59:16.062347+0000","last_clean":"2024-09-13T04:59:16.062347+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:59:16.062347+0000","last_undegraded":"2024-09-13T04:59:16.062347+0000","last_fullsized":"2024-09-13T04:59:16.062347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":30190097,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":2318,"num_read_kb":17915,"num_write":510,"num_write_kb":5973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"75'295","reported_seq":822,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.063002+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:59:16.063002+0000","last_peered":"2024-09-13T04:59:16.063002+0000","last_clean":"2024-09-13T04:59:16.063002+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:59:16.063002+0000","last_undegraded":"2024-09-13T04:59:16.063002+0000","last_fullsized":"2024-09-13T04:59:16.063002+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":17767641,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":1343,"num_read_kb":9470,"num_write":347,"num_write_kb":5144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"76'551","reported_seq":3166,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.090920+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:59:16.090920+0000","last_peered":"2024-09-13T04:59:16.090920+0000","last_clean":"2024-09-13T04:59:16.090920+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:59:16.090920+0000","last_undegraded":"2024-09-13T04:59:16.090920+0000","last_fullsized":"2024-09-13T04:59:16.090920+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":19458231,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":28,"num_read":4628,"num_read_kb":12719,"num_write":784,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091277+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:59:16.091277+0000","last_peered":"2024-09-13T04:59:16.091277+0000","last_clean":"2024-09-13T04:59:16.091277+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:59:16.091277+0000","last_undegraded":"2024-09-13T04:59:16.091277+0000","last_fullsized":"2024-09-13T04:59:16.091277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"75'194","reported_seq":2291,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.064302+0000","last_change":"2024-09-13T04:58:19.436873+0000","last_active":"2024-09-13T04:59:16.064302+0000","last_peered":"2024-09-13T04:59:16.064302+0000","last_clean":"2024-09-13T04:59:16.064302+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T04:59:16.064302+0000","last_undegraded":"2024-09-13T04:59:16.064302+0000","last_fullsized":"2024-09-13T04:59:16.064302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":47066263,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":2130,"num_read_kb":23085,"num_write":213,"num_write_kb":19550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.092261+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:59:16.092261+0000","last_peered":"2024-09-13T04:59:16.092261+0000","last_clean":"2024-09-13T04:59:16.092261+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:59:16.092261+0000","last_undegraded":"2024-09-13T04:59:16.092261+0000","last_fullsized":"2024-09-13T04:59:16.092261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"75'277","reported_seq":1389,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.092363+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:59:16.092363+0000","last_peered":"2024-09-13T04:59:16.092363+0000","last_clean":"2024-09-13T04:59:16.092363+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:59:16.092363+0000","last_undegraded":"2024-09-13T04:59:16.092363+0000","last_fullsized":"2024-09-13T04:59:16.092363+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":18650138,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":2382,"num_read_kb":10650,"num_write":306,"num_write_kb":4953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091305+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:59:16.091305+0000","last_peered":"2024-09-13T04:59:16.091305+0000","last_clean":"2024-09-13T04:59:16.091305+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:59:16.091305+0000","last_undegraded":"2024-09-13T04:59:16.091305+0000","last_fullsized":"2024-09-13T04:59:16.091305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"74'443","reported_seq":933,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.064092+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:59:16.064092+0000","last_peered":"2024-09-13T04:59:16.064092+0000","last_clean":"2024-09-13T04:59:16.064092+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:59:16.064092+0000","last_undegraded":"2024-09-13T04:59:16.064092+0000","last_fullsized":"2024-09-13T04:59:16.064092+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":19331032,"num_objects":59,"num_object_clones":9,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":1832,"num_read_kb":7525,"num_write":637,"num_write_kb":5512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T04:59:15.073330+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:59:15.073330+0000","last_peered":"2024-09-13T04:59:15.073330+0000","last_clean":"2024-09-13T04:59:15.073330+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:59:15.073330+0000","last_undegraded":"2024-09-13T04:59:15.073330+0000","last_fullsized":"2024-09-13T04:59:15.073330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"79'400","reported_seq":1364,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.406069+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:59:17.406069+0000","last_peered":"2024-09-13T04:59:17.406069+0000","last_clean":"2024-09-13T04:59:17.406069+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:59:17.406069+0000","last_undegraded":"2024-09-13T04:59:17.406069+0000","last_fullsized":"2024-09-13T04:59:17.406069+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":18100282,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":2867,"num_read_kb":10200,"num_write":468,"num_write_kb":6005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"79'449","reported_seq":3529,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.664192+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:59:17.664192+0000","last_peered":"2024-09-13T04:59:17.664192+0000","last_clean":"2024-09-13T04:59:17.664192+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:59:17.664192+0000","last_undegraded":"2024-09-13T04:59:17.664192+0000","last_fullsized":"2024-09-13T04:59:17.664192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":49756128,"num_objects":73,"num_object_clones":20,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":5049,"num_read_kb":20398,"num_write":497,"num_write_kb":14722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":74,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.090992+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:59:16.090992+0000","last_peered":"2024-09-13T04:59:16.090992+0000","last_clean":"2024-09-13T04:59:16.090992+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:59:16.090992+0000","last_undegraded":"2024-09-13T04:59:16.090992+0000","last_fullsized":"2024-09-13T04:59:16.090992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"78'471","reported_seq":1262,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.474661+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:59:16.474661+0000","last_peered":"2024-09-13T04:59:16.474661+0000","last_clean":"2024-09-13T04:59:16.474661+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:59:16.474661+0000","last_undegraded":"2024-09-13T04:59:16.474661+0000","last_fullsized":"2024-09-13T04:59:16.474661+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":31585551,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":2716,"num_read_kb":9859,"num_write":627,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091211+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:59:16.091211+0000","last_peered":"2024-09-13T04:59:16.091211+0000","last_clean":"2024-09-13T04:59:16.091211+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:59:16.091211+0000","last_undegraded":"2024-09-13T04:59:16.091211+0000","last_fullsized":"2024-09-13T04:59:16.091211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'283","reported_seq":4516,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.295270+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:59:17.295270+0000","last_peered":"2024-09-13T04:59:17.295270+0000","last_clean":"2024-09-13T04:59:17.295270+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:59:17.295270+0000","last_undegraded":"2024-09-13T04:59:17.295270+0000","last_fullsized":"2024-09-13T04:59:17.295270+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":40165106,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":5230,"num_read_kb":22401,"num_write":272,"num_write_kb":6643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.090787+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:59:16.090787+0000","last_peered":"2024-09-13T04:59:16.090787+0000","last_clean":"2024-09-13T04:59:16.090787+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:59:16.090787+0000","last_undegraded":"2024-09-13T04:59:16.090787+0000","last_fullsized":"2024-09-13T04:59:16.090787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"75'645","reported_seq":1939,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091973+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:59:16.091973+0000","last_peered":"2024-09-13T04:59:16.091973+0000","last_clean":"2024-09-13T04:59:16.091973+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:59:16.091973+0000","last_undegraded":"2024-09-13T04:59:16.091973+0000","last_fullsized":"2024-09-13T04:59:16.091973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":31149402,"num_objects":51,"num_object_clones":5,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":23,"num_read":4730,"num_read_kb":23406,"num_write":950,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091228+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:59:16.091228+0000","last_peered":"2024-09-13T04:59:16.091228+0000","last_clean":"2024-09-13T04:59:16.091228+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:59:16.091228+0000","last_undegraded":"2024-09-13T04:59:16.091228+0000","last_fullsized":"2024-09-13T04:59:16.091228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":219,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.086796+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:59:16.086796+0000","last_peered":"2024-09-13T04:59:16.086796+0000","last_clean":"2024-09-13T04:59:16.086796+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:59:16.086796+0000","last_undegraded":"2024-09-13T04:59:16.086796+0000","last_fullsized":"2024-09-13T04:59:16.086796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"79'307","reported_seq":1142,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.397645+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:59:17.397645+0000","last_peered":"2024-09-13T04:59:17.397645+0000","last_clean":"2024-09-13T04:59:17.397645+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:59:17.397645+0000","last_undegraded":"2024-09-13T04:59:17.397645+0000","last_fullsized":"2024-09-13T04:59:17.397645+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":37593098,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":2150,"num_read_kb":4644,"num_write":347,"num_write_kb":2864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"79'458","reported_seq":5385,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.435028+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:59:17.435028+0000","last_peered":"2024-09-13T04:59:17.435028+0000","last_clean":"2024-09-13T04:59:17.435028+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:59:17.435028+0000","last_undegraded":"2024-09-13T04:59:17.435028+0000","last_fullsized":"2024-09-13T04:59:17.435028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":26098146,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":27,"num_read":6422,"num_read_kb":21563,"num_write":554,"num_write_kb":10268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"78'291","reported_seq":5519,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.494613+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:59:16.494613+0000","last_peered":"2024-09-13T04:59:16.494613+0000","last_clean":"2024-09-13T04:59:16.494613+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:59:16.494613+0000","last_undegraded":"2024-09-13T04:59:16.494613+0000","last_fullsized":"2024-09-13T04:59:16.494613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":33438901,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":25,"num_read":5449,"num_read_kb":23414,"num_write":300,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"76'206","reported_seq":771,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091218+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:59:16.091218+0000","last_peered":"2024-09-13T04:59:16.091218+0000","last_clean":"2024-09-13T04:59:16.091218+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:59:16.091218+0000","last_undegraded":"2024-09-13T04:59:16.091218+0000","last_fullsized":"2024-09-13T04:59:16.091218+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":32911942,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":26,"num_read":595,"num_read_kb":15992,"num_write":204,"num_write_kb":9206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"79'493","reported_seq":1007,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.299211+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:59:17.299211+0000","last_peered":"2024-09-13T04:59:17.299211+0000","last_clean":"2024-09-13T04:59:17.299211+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:59:17.299211+0000","last_undegraded":"2024-09-13T04:59:17.299211+0000","last_fullsized":"2024-09-13T04:59:17.299211+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":46629385,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":33,"num_read":1925,"num_read_kb":7115,"num_write":706,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"74'317","reported_seq":3934,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091081+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:59:16.091081+0000","last_peered":"2024-09-13T04:59:16.091081+0000","last_clean":"2024-09-13T04:59:16.091081+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:59:16.091081+0000","last_undegraded":"2024-09-13T04:59:16.091081+0000","last_fullsized":"2024-09-13T04:59:16.091081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":36812324,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":5286,"num_read_kb":33748,"num_write":490,"num_write_kb":12841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"75'197","reported_seq":667,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.062410+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:59:16.062410+0000","last_peered":"2024-09-13T04:59:16.062410+0000","last_clean":"2024-09-13T04:59:16.062410+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:59:16.062410+0000","last_undegraded":"2024-09-13T04:59:16.062410+0000","last_fullsized":"2024-09-13T04:59:16.062410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":18593565,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":21,"num_read":773,"num_read_kb":6573,"num_write":181,"num_write_kb":3323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"79'683","reported_seq":5818,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.646983+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:59:17.646983+0000","last_peered":"2024-09-13T04:59:17.646983+0000","last_clean":"2024-09-13T04:59:17.646983+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:59:17.646983+0000","last_undegraded":"2024-09-13T04:59:17.646983+0000","last_fullsized":"2024-09-13T04:59:17.646983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":23509628,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":33,"num_read":7592,"num_read_kb":21669,"num_write":1009,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"79'283","reported_seq":4501,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.527489+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:59:17.527489+0000","last_peered":"2024-09-13T04:59:17.527489+0000","last_clean":"2024-09-13T04:59:17.527489+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:59:17.527489+0000","last_undegraded":"2024-09-13T04:59:17.527489+0000","last_fullsized":"2024-09-13T04:59:17.527489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":37645132,"num_objects":73,"num_object_clones":19,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":38,"num_read":4960,"num_read_kb":10511,"num_write":281,"num_write_kb":3309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"75'294","reported_seq":4722,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.090884+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:59:16.090884+0000","last_peered":"2024-09-13T04:59:16.090884+0000","last_clean":"2024-09-13T04:59:16.090884+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:59:16.090884+0000","last_undegraded":"2024-09-13T04:59:16.090884+0000","last_fullsized":"2024-09-13T04:59:16.090884+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":55813768,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":5174,"num_read_kb":31153,"num_write":305,"num_write_kb":16052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"76'227","reported_seq":2122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.064216+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:59:16.064216+0000","last_peered":"2024-09-13T04:59:16.064216+0000","last_clean":"2024-09-13T04:59:16.064216+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:59:16.064216+0000","last_undegraded":"2024-09-13T04:59:16.064216+0000","last_fullsized":"2024-09-13T04:59:16.064216+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":18466177,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":2003,"num_read_kb":11259,"num_write":242,"num_write_kb":6350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"79'481","reported_seq":1357,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.597011+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:59:17.597011+0000","last_peered":"2024-09-13T04:59:17.597011+0000","last_clean":"2024-09-13T04:59:17.597011+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:59:17.597011+0000","last_undegraded":"2024-09-13T04:59:17.597011+0000","last_fullsized":"2024-09-13T04:59:17.597011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":16758913,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":2853,"num_read_kb":8923,"num_write":658,"num_write_kb":4712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1006793677,"num_objects":1769,"num_object_clones":417,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":828,"num_read":120532,"num_read_kb":565341,"num_write":14106,"num_write_kb":270641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":52068352,"data_stored":413050441,"data_compressed":16161466,"data_compressed_allocated":40534016,"data_compressed_original":404809876,"omap_allocated":0,"internal_metadata":0},"log_size":11379,"ondisk_log_size":11379,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38992,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332848,"statfs":{"total":96636764160,"available":96596836352,"internally_reserved":0,"allocated":3866624,"data_stored":20448781,"data_compressed":1003423,"data_compressed_allocated":2183168,"data_compressed_original":19537643,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.747},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.95899999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.86299999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.05}]}]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46280,"kb_used_data":6972,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325560,"statfs":{"total":96636764160,"available":96589373440,"internally_reserved":0,"allocated":7139328,"data_stored":53659887,"data_compressed":1580821,"data_compressed_allocated":4751360,"data_compressed_original":52204328,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.87},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":1.089},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.628}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.016},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.92400000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46520,"kb_used_data":7216,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325320,"statfs":{"total":96636764160,"available":96589127680,"internally_reserved":0,"allocated":7389184,"data_stored":53246929,"data_compressed":2004862,"data_compressed_allocated":5341184,"data_compressed_original":52053963,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.749}]},{"osd":1,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.86199999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":11.266,"5min":11.266,"15min":11.266},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.436},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.57199999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36520,"kb_used_data":9496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335320,"statfs":{"total":96636764160,"available":96599367680,"internally_reserved":0,"allocated":9723904,"data_stored":68404082,"data_compressed":2795877,"data_compressed_allocated":6873088,"data_compressed_original":66573031,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.25},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.30499999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.33900000000000002}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.629},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.876}]}]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34212,"kb_used_data":7192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337628,"statfs":{"total":96636764160,"available":96601731072,"internally_reserved":0,"allocated":7364608,"data_stored":57228471,"data_compressed":2155328,"data_compressed_allocated":5103616,"data_compressed_original":55888312,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.434},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.374}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.496}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.746}]}]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36508,"kb_used_data":9492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":9719808,"data_stored":68773774,"data_compressed":2460604,"data_compressed_allocated":7094272,"data_compressed_original":67099820,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.028}]},{"osd":2,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84199999999999997}]},{"osd":6,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.121},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.97199999999999998}]},{"osd":7,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.0649999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34332,"kb_used_data":7316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337508,"statfs":{"total":96636764160,"available":96601608192,"internally_reserved":0,"allocated":7491584,"data_stored":49597562,"data_compressed":2084970,"data_compressed_allocated":4861952,"data_compressed_original":47946012,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.21199999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":15.755000000000001,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":15.802,"5min":15.802,"15min":15.802},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1,"5min":1,"15min":1},"last":2.915},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":1.794,"15min":1.794},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":15.859999999999999,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":3.109}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":11.474,"5min":11.474,"15min":11.474},"last":3.1659999999999999},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":11.528,"5min":11.528,"15min":11.528},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.897,"5min":1.897,"15min":1.897},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":15.885999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":15.834,"5min":15.834,"15min":15.834},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":3.0670000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46108,"kb_used_data":6800,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325732,"statfs":{"total":96636764160,"available":96589549568,"internally_reserved":0,"allocated":6963200,"data_stored":46291735,"data_compressed":2196272,"data_compressed_allocated":4517888,"data_compressed_original":44768335,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.627},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.633,"5min":4.633,"15min":4.633},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.753},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.6989999999999998,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.0660000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.008},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.84299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6557696,"data_stored":49211526,"data_compressed":2066120,"data_compressed_allocated":4845568,"data_compressed_original":47974684,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8667136,"data_stored":67676505,"data_compressed":2400118,"data_compressed_allocated":7000064,"data_compressed_original":66444460,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6443008,"data_stored":56813178,"data_compressed":2155328,"data_compressed_allocated":5103616,"data_compressed_original":55888312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8736768,"data_stored":67878381,"data_compressed":2760531,"data_compressed_allocated":6823936,"data_compressed_original":66462439,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6467584,"data_stored":52831636,"data_compressed":2004862,"data_compressed_allocated":5341184,"data_compressed_original":52053963,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6000640,"data_stored":45338084,"data_compressed":2189939,"data_compressed_allocated":4481024,"data_compressed_original":44231759,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6209536,"data_stored":53206320,"data_compressed":1580752,"data_compressed_allocated":4747264,"data_compressed_original":52167464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2985984,"data_stored":20094811,"data_compressed":1003816,"data_compressed_allocated":2191360,"data_compressed_original":19586795,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:59:21.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:59:21.535 INFO:teuthology.orchestra.run.smithi001.stdout:5040 write 0xd6a3f70 thru 0xd6b14a4 (0xd535 bytes) 2024-09-13T04:59:21.535 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.535 INFO:teuthology.orchestra.run.smithi001.stdout:5044 read 0x4215578 thru 0x421ee06 (0x988f bytes) 2024-09-13T04:59:21.535 INFO:teuthology.orchestra.run.smithi001.stdout:5045 write 0xc3d419b thru 0xc3ded76 (0xabdc bytes) 2024-09-13T04:59:21.536 INFO:teuthology.orchestra.run.smithi001.stdout:5046 writesame 0xb489ddd thru 0xb49361a data_size 0x2(0x983e bytes) 2024-09-13T04:59:21.536 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.536 INFO:teuthology.orchestra.run.smithi001.stdout:5050 writesame 0x6a414b6 thru 0x6a41eda data_size 0x7(0xa25 bytes) 2024-09-13T04:59:21.536 INFO:teuthology.orchestra.run.smithi001.stdout:5051 read 0x16d2a79 thru 0x16d4646 (0x1bce bytes) 2024-09-13T04:59:21.536 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.536 INFO:teuthology.orchestra.run.smithi001.stdout:5053 trunc from 0xd6b14a5 to 0xdd67e46 2024-09-13T04:59:21.541 INFO:teuthology.orchestra.run.smithi001.stdout:5056 write 0xb3ab10c thru 0xb3b199c (0x6891 bytes) 2024-09-13T04:59:21.556 INFO:teuthology.orchestra.run.smithi001.stdout:5057 trunc from 0xdd67e46 to 0x971fbe5 2024-09-13T04:59:21.573 INFO:teuthology.orchestra.run.smithi001.stdout:5058 read 0x74391d3 thru 0x743dc2b (0x4a59 bytes) 2024-09-13T04:59:21.580 INFO:teuthology.orchestra.run.smithi001.stdout:5059 trunc from 0x971fbe5 to 0x4bcf6c5 2024-09-13T04:59:21.626 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:21.622+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:21.627 INFO:teuthology.orchestra.run.smithi001.stdout:5062 writesame 0xaa0ddac thru 0xaa12dbb data_size 0x70(0x5010 bytes) 2024-09-13T04:59:21.627 INFO:teuthology.orchestra.run.smithi001.stdout:5063 writesame 0x279e896 thru 0x27a0939 data_size 0x2(0x20a4 bytes) 2024-09-13T04:59:21.635 INFO:teuthology.orchestra.run.smithi001.stdout:5064 punch from 0x152b8d to 0x157cc1, (0x5134 bytes) 2024-09-13T04:59:21.636 INFO:teuthology.orchestra.run.smithi001.stdout:5065 punch from 0x6e24a83 to 0x6e2c272, (0x77ef bytes) 2024-09-13T04:59:21.636 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.636 INFO:teuthology.orchestra.run.smithi001.stdout:5067 writesame 0x12a74bf thru 0x12b0f41 data_size 0xf(0x9a83 bytes) 2024-09-13T04:59:21.641 INFO:teuthology.orchestra.run.smithi001.stdout:5068 read 0x4010c7f thru 0x40117c9 (0xb4b bytes) 2024-09-13T04:59:21.641 INFO:teuthology.orchestra.run.smithi001.stdout:5069 write 0xc8b2f thru 0xd18c2 (0x8d94 bytes) 2024-09-13T04:59:21.687 INFO:teuthology.orchestra.run.smithi001.stdout:5071 write 0xc05bf34 thru 0xc065b59 (0x9c26 bytes) 2024-09-13T04:59:21.691 INFO:teuthology.orchestra.run.smithi001.stdout:5072 writesame 0x51289a9 thru 0x51328dc data_size 0x17(0x9f34 bytes) 2024-09-13T04:59:21.698 INFO:teuthology.orchestra.run.smithi001.stdout:5073 trunc from 0xc065b5a to 0xaee129d 2024-09-13T04:59:21.701 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:21.701 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:21.701 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:21.701 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:21.714 INFO:teuthology.orchestra.run.smithi001.stdout:5075 write 0x7cf3592 thru 0x7cf35eb (0x5a bytes) 2024-09-13T04:59:21.719 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:21.720 INFO:teuthology.orchestra.run.smithi001.stdout:5076 punch from 0x6cc58c5 to 0x6cd2770, (0xceab bytes) 2024-09-13T04:59:21.720 INFO:teuthology.orchestra.run.smithi001.stdout:5077 punch from 0xae94e1b to 0xae98b77, (0x3d5c bytes) 2024-09-13T04:59:21.720 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.720 INFO:teuthology.orchestra.run.smithi001.stdout:5081 write 0x855870f thru 0x855f664 (0x6f56 bytes) 2024-09-13T04:59:21.725 INFO:teuthology.orchestra.run.smithi001.stdout:5082 punch from 0x2f47a3f to 0x2f4ef51, (0x7512 bytes) 2024-09-13T04:59:21.725 INFO:teuthology.orchestra.run.smithi001.stdout:5083 write 0xa0052f6 thru 0xa00f5ee (0xa2f9 bytes) 2024-09-13T04:59:21.732 INFO:teuthology.orchestra.run.smithi001.stdout:5085 read 0xfef227 thru 0xff3baa (0x4984 bytes) 2024-09-13T04:59:21.732 INFO:teuthology.orchestra.run.smithi001.stdout:5086 read 0x746ed9b thru 0x747d357 (0xe5bd bytes) 2024-09-13T04:59:21.732 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.732 INFO:teuthology.orchestra.run.smithi001.stdout:5088 read 0x359073d thru 0x359ee79 (0xe73d bytes) 2024-09-13T04:59:21.739 INFO:teuthology.orchestra.run.smithi001.stdout:5089 trunc from 0xaee129d to 0x466a881 2024-09-13T04:59:21.758 INFO:teuthology.orchestra.run.smithi001.stdout:5090 punch from 0x31bd9d3 to 0x31c7514, (0x9b41 bytes) 2024-09-13T04:59:21.758 INFO:teuthology.orchestra.run.smithi001.stdout:5092 read 0x17e439b thru 0x17ed4af (0x9115 bytes) 2024-09-13T04:59:21.759 INFO:teuthology.orchestra.run.smithi001.stdout:5093 trunc from 0x466a881 to 0x56323e6 2024-09-13T04:59:21.764 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:21.762+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:21.771 INFO:teuthology.orchestra.run.smithi001.stdout:5094 read 0x4d93edf thru 0x4da3524 (0xf646 bytes) 2024-09-13T04:59:21.772 INFO:teuthology.orchestra.run.smithi001.stdout:5095 punch from 0x3310287 to 0x3316b77, (0x68f0 bytes) 2024-09-13T04:59:21.772 INFO:teuthology.orchestra.run.smithi001.stdout:5097 write 0x327394e thru 0x328105d (0xd710 bytes) 2024-09-13T04:59:21.786 INFO:teuthology.orchestra.run.smithi001.stdout:5100 punch from 0x35adf9b to 0x35b8e68, (0xaecd bytes) 2024-09-13T04:59:21.786 INFO:teuthology.orchestra.run.smithi001.stdout:5101 read 0x311233d thru 0x31138ba (0x157e bytes) 2024-09-13T04:59:21.801 INFO:teuthology.orchestra.run.smithi001.stdout:5102 write 0x710e9d7 thru 0x711950b (0xab35 bytes) 2024-09-13T04:59:21.809 INFO:teuthology.orchestra.run.smithi001.stdout:5103 read 0x13a2c7e thru 0x13b135a (0xe6dd bytes) 2024-09-13T04:59:21.830 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:21.826+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:5105 writesame 0xea86b8c thru 0xea94ae7 data_size 0x1e(0xdf5c bytes) 2024-09-13T04:59:21.836 INFO:teuthology.orchestra.run.smithi001.stdout:5106 writesame 0x42ee8c9 thru 0x42fa606 data_size 0x1(0xbd3e bytes) 2024-09-13T04:59:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:59:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:59:21.859 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-13T04:59:21.104505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1007384064,"num_objects":1773,"num_object_clones":417,"num_object_copies":3546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1773,"num_whiteouts":828,"num_read":120648,"num_read_kb":565441,"num_write":14267,"num_write_kb":273059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11499,"ondisk_log_size":11499,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":319472,"kb_used_data":58260,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754655248,"statfs":{"total":773094113280,"available":772766973952,"internally_reserved":0,"allocated":59658240,"data_stored":417651221,"data_compressed":16282157,"data_compressed_allocated":40726528,"data_compressed_original":406071444,"omap_allocated":12743,"internal_metadata":267374137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19683856,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":3667,"num_read_kb":21130,"num_write":857,"num_write_kb":7669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035219"},"pg_stats":[{"pgid":"3.1f","version":"77'389","reported_seq":2607,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T04:59:15.351646+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:59:15.351646+0000","last_peered":"2024-09-13T04:59:15.351646+0000","last_clean":"2024-09-13T04:59:15.351646+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:59:15.351646+0000","last_undegraded":"2024-09-13T04:59:15.351646+0000","last_fullsized":"2024-09-13T04:59:15.351646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":51629431,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":3731,"num_read_kb":15540,"num_write":438,"num_write_kb":8468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"76'231","reported_seq":1709,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T04:59:15.073332+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:59:15.073332+0000","last_peered":"2024-09-13T04:59:15.073332+0000","last_clean":"2024-09-13T04:59:15.073332+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:59:15.073332+0000","last_undegraded":"2024-09-13T04:59:15.073332+0000","last_fullsized":"2024-09-13T04:59:15.073332+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":17572659,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":2351,"num_read_kb":14458,"num_write":218,"num_write_kb":5747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"79'417","reported_seq":1084,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.491355+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:59:17.491355+0000","last_peered":"2024-09-13T04:59:17.491355+0000","last_clean":"2024-09-13T04:59:17.491355+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:59:17.491355+0000","last_undegraded":"2024-09-13T04:59:17.491355+0000","last_fullsized":"2024-09-13T04:59:17.491355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":48762137,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":31,"num_read":1743,"num_read_kb":13131,"num_write":576,"num_write_kb":8676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"76'316","reported_seq":984,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.086848+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:59:16.086848+0000","last_peered":"2024-09-13T04:59:16.086848+0000","last_clean":"2024-09-13T04:59:16.086848+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:59:16.086848+0000","last_undegraded":"2024-09-13T04:59:16.086848+0000","last_fullsized":"2024-09-13T04:59:16.086848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":30471990,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":1231,"num_read_kb":6683,"num_write":364,"num_write_kb":6514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'341","reported_seq":4603,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.092031+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:59:16.092031+0000","last_peered":"2024-09-13T04:59:16.092031+0000","last_clean":"2024-09-13T04:59:16.092031+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:59:16.092031+0000","last_undegraded":"2024-09-13T04:59:16.092031+0000","last_fullsized":"2024-09-13T04:59:16.092031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":25770403,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":5294,"num_read_kb":15720,"num_write":393,"num_write_kb":5849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"79'180","reported_seq":2061,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.576386+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:59:17.576386+0000","last_peered":"2024-09-13T04:59:17.576386+0000","last_clean":"2024-09-13T04:59:17.576386+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:59:17.576386+0000","last_undegraded":"2024-09-13T04:59:17.576386+0000","last_fullsized":"2024-09-13T04:59:17.576386+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":24036588,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":1961,"num_read_kb":10026,"num_write":185,"num_write_kb":7409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"78'335","reported_seq":17576,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.495531+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:59:16.495531+0000","last_peered":"2024-09-13T04:59:16.495531+0000","last_clean":"2024-09-13T04:59:16.495531+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:59:16.495531+0000","last_undegraded":"2024-09-13T04:59:16.495531+0000","last_fullsized":"2024-09-13T04:59:16.495531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":46179149,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":18227,"num_read_kb":95531,"num_write":398,"num_write_kb":18454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"72'333","reported_seq":2469,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T04:59:15.073608+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:59:15.073608+0000","last_peered":"2024-09-13T04:59:15.073608+0000","last_clean":"2024-09-13T04:59:15.073608+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:59:15.073608+0000","last_undegraded":"2024-09-13T04:59:15.073608+0000","last_fullsized":"2024-09-13T04:59:15.073608+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":26896124,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":31,"num_read":2905,"num_read_kb":8958,"num_write":422,"num_write_kb":5185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"78'211","reported_seq":2899,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.543883+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:59:16.543883+0000","last_peered":"2024-09-13T04:59:16.543883+0000","last_clean":"2024-09-13T04:59:16.543883+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:59:16.543883+0000","last_undegraded":"2024-09-13T04:59:16.543883+0000","last_fullsized":"2024-09-13T04:59:16.543883+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":27984346,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":2682,"num_read_kb":21102,"num_write":224,"num_write_kb":13793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"75'381","reported_seq":1685,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.062347+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:59:16.062347+0000","last_peered":"2024-09-13T04:59:16.062347+0000","last_clean":"2024-09-13T04:59:16.062347+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:59:16.062347+0000","last_undegraded":"2024-09-13T04:59:16.062347+0000","last_fullsized":"2024-09-13T04:59:16.062347+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":30190097,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":2318,"num_read_kb":17915,"num_write":510,"num_write_kb":5973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"75'295","reported_seq":822,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.063002+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:59:16.063002+0000","last_peered":"2024-09-13T04:59:16.063002+0000","last_clean":"2024-09-13T04:59:16.063002+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:59:16.063002+0000","last_undegraded":"2024-09-13T04:59:16.063002+0000","last_fullsized":"2024-09-13T04:59:16.063002+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":17767641,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":1343,"num_read_kb":9470,"num_write":347,"num_write_kb":5144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"76'551","reported_seq":3166,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.090920+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:59:16.090920+0000","last_peered":"2024-09-13T04:59:16.090920+0000","last_clean":"2024-09-13T04:59:16.090920+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:59:16.090920+0000","last_undegraded":"2024-09-13T04:59:16.090920+0000","last_fullsized":"2024-09-13T04:59:16.090920+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":19458231,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":28,"num_read":4628,"num_read_kb":12719,"num_write":784,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091277+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:59:16.091277+0000","last_peered":"2024-09-13T04:59:16.091277+0000","last_clean":"2024-09-13T04:59:16.091277+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:59:16.091277+0000","last_undegraded":"2024-09-13T04:59:16.091277+0000","last_fullsized":"2024-09-13T04:59:16.091277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"75'194","reported_seq":2291,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.064302+0000","last_change":"2024-09-13T04:58:19.436873+0000","last_active":"2024-09-13T04:59:16.064302+0000","last_peered":"2024-09-13T04:59:16.064302+0000","last_clean":"2024-09-13T04:59:16.064302+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T04:59:16.064302+0000","last_undegraded":"2024-09-13T04:59:16.064302+0000","last_fullsized":"2024-09-13T04:59:16.064302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":47066263,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":2130,"num_read_kb":23085,"num_write":213,"num_write_kb":19550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.092261+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:59:16.092261+0000","last_peered":"2024-09-13T04:59:16.092261+0000","last_clean":"2024-09-13T04:59:16.092261+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:59:16.092261+0000","last_undegraded":"2024-09-13T04:59:16.092261+0000","last_fullsized":"2024-09-13T04:59:16.092261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"75'277","reported_seq":1389,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.092363+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:59:16.092363+0000","last_peered":"2024-09-13T04:59:16.092363+0000","last_clean":"2024-09-13T04:59:16.092363+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:59:16.092363+0000","last_undegraded":"2024-09-13T04:59:16.092363+0000","last_fullsized":"2024-09-13T04:59:16.092363+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":18650138,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":2382,"num_read_kb":10650,"num_write":306,"num_write_kb":4953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091305+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:59:16.091305+0000","last_peered":"2024-09-13T04:59:16.091305+0000","last_clean":"2024-09-13T04:59:16.091305+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:59:16.091305+0000","last_undegraded":"2024-09-13T04:59:16.091305+0000","last_fullsized":"2024-09-13T04:59:16.091305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"74'443","reported_seq":933,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.064092+0000","last_change":"2024-09-13T04:58:05.976297+0000","last_active":"2024-09-13T04:59:16.064092+0000","last_peered":"2024-09-13T04:59:16.064092+0000","last_clean":"2024-09-13T04:59:16.064092+0000","last_became_active":"2024-09-13T04:58:03.931483+0000","last_became_peered":"2024-09-13T04:58:03.931483+0000","last_unstale":"2024-09-13T04:59:16.064092+0000","last_undegraded":"2024-09-13T04:59:16.064092+0000","last_fullsized":"2024-09-13T04:59:16.064092+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:44.226892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":19331032,"num_objects":59,"num_object_clones":9,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":1832,"num_read_kb":7525,"num_write":637,"num_write_kb":5512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-13T04:59:15.073330+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:59:15.073330+0000","last_peered":"2024-09-13T04:59:15.073330+0000","last_clean":"2024-09-13T04:59:15.073330+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:59:15.073330+0000","last_undegraded":"2024-09-13T04:59:15.073330+0000","last_fullsized":"2024-09-13T04:59:15.073330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"79'400","reported_seq":1364,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.406069+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:59:17.406069+0000","last_peered":"2024-09-13T04:59:17.406069+0000","last_clean":"2024-09-13T04:59:17.406069+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:59:17.406069+0000","last_undegraded":"2024-09-13T04:59:17.406069+0000","last_fullsized":"2024-09-13T04:59:17.406069+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":18100282,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":26,"num_read":2867,"num_read_kb":10200,"num_write":468,"num_write_kb":6005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"79'449","reported_seq":3529,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.664192+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:59:17.664192+0000","last_peered":"2024-09-13T04:59:17.664192+0000","last_clean":"2024-09-13T04:59:17.664192+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:59:17.664192+0000","last_undegraded":"2024-09-13T04:59:17.664192+0000","last_fullsized":"2024-09-13T04:59:17.664192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":49756128,"num_objects":73,"num_object_clones":20,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":5049,"num_read_kb":20398,"num_write":497,"num_write_kb":14722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":74,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.090992+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:59:16.090992+0000","last_peered":"2024-09-13T04:59:16.090992+0000","last_clean":"2024-09-13T04:59:16.090992+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:59:16.090992+0000","last_undegraded":"2024-09-13T04:59:16.090992+0000","last_fullsized":"2024-09-13T04:59:16.090992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"78'471","reported_seq":1262,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.474661+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:59:16.474661+0000","last_peered":"2024-09-13T04:59:16.474661+0000","last_clean":"2024-09-13T04:59:16.474661+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:59:16.474661+0000","last_undegraded":"2024-09-13T04:59:16.474661+0000","last_fullsized":"2024-09-13T04:59:16.474661+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":31585551,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":2716,"num_read_kb":9859,"num_write":627,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091211+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:59:16.091211+0000","last_peered":"2024-09-13T04:59:16.091211+0000","last_clean":"2024-09-13T04:59:16.091211+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:59:16.091211+0000","last_undegraded":"2024-09-13T04:59:16.091211+0000","last_fullsized":"2024-09-13T04:59:16.091211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'283","reported_seq":4516,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.295270+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:59:17.295270+0000","last_peered":"2024-09-13T04:59:17.295270+0000","last_clean":"2024-09-13T04:59:17.295270+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:59:17.295270+0000","last_undegraded":"2024-09-13T04:59:17.295270+0000","last_fullsized":"2024-09-13T04:59:17.295270+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":40165106,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":5230,"num_read_kb":22401,"num_write":272,"num_write_kb":6643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.090787+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:59:16.090787+0000","last_peered":"2024-09-13T04:59:16.090787+0000","last_clean":"2024-09-13T04:59:16.090787+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:59:16.090787+0000","last_undegraded":"2024-09-13T04:59:16.090787+0000","last_fullsized":"2024-09-13T04:59:16.090787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"75'645","reported_seq":1939,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091973+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:59:16.091973+0000","last_peered":"2024-09-13T04:59:16.091973+0000","last_clean":"2024-09-13T04:59:16.091973+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:59:16.091973+0000","last_undegraded":"2024-09-13T04:59:16.091973+0000","last_fullsized":"2024-09-13T04:59:16.091973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":31149402,"num_objects":51,"num_object_clones":5,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":23,"num_read":4730,"num_read_kb":23406,"num_write":950,"num_write_kb":10933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091228+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:59:16.091228+0000","last_peered":"2024-09-13T04:59:16.091228+0000","last_clean":"2024-09-13T04:59:16.091228+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:59:16.091228+0000","last_undegraded":"2024-09-13T04:59:16.091228+0000","last_fullsized":"2024-09-13T04:59:16.091228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":219,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.086796+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:59:16.086796+0000","last_peered":"2024-09-13T04:59:16.086796+0000","last_clean":"2024-09-13T04:59:16.086796+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:59:16.086796+0000","last_undegraded":"2024-09-13T04:59:16.086796+0000","last_fullsized":"2024-09-13T04:59:16.086796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"79'307","reported_seq":1142,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.397645+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:59:17.397645+0000","last_peered":"2024-09-13T04:59:17.397645+0000","last_clean":"2024-09-13T04:59:17.397645+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:59:17.397645+0000","last_undegraded":"2024-09-13T04:59:17.397645+0000","last_fullsized":"2024-09-13T04:59:17.397645+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":37593098,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":2150,"num_read_kb":4644,"num_write":347,"num_write_kb":2864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"79'458","reported_seq":5385,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.435028+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:59:17.435028+0000","last_peered":"2024-09-13T04:59:17.435028+0000","last_clean":"2024-09-13T04:59:17.435028+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:59:17.435028+0000","last_undegraded":"2024-09-13T04:59:17.435028+0000","last_fullsized":"2024-09-13T04:59:17.435028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":26098146,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":27,"num_read":6422,"num_read_kb":21563,"num_write":554,"num_write_kb":10268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"78'291","reported_seq":5519,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.494613+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:59:16.494613+0000","last_peered":"2024-09-13T04:59:16.494613+0000","last_clean":"2024-09-13T04:59:16.494613+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:59:16.494613+0000","last_undegraded":"2024-09-13T04:59:16.494613+0000","last_fullsized":"2024-09-13T04:59:16.494613+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":33438901,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":25,"num_read":5449,"num_read_kb":23414,"num_write":300,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"76'206","reported_seq":771,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091218+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:59:16.091218+0000","last_peered":"2024-09-13T04:59:16.091218+0000","last_clean":"2024-09-13T04:59:16.091218+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:59:16.091218+0000","last_undegraded":"2024-09-13T04:59:16.091218+0000","last_fullsized":"2024-09-13T04:59:16.091218+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":32911942,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":26,"num_read":595,"num_read_kb":15992,"num_write":204,"num_write_kb":9206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"79'493","reported_seq":1007,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.299211+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:59:17.299211+0000","last_peered":"2024-09-13T04:59:17.299211+0000","last_clean":"2024-09-13T04:59:17.299211+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:59:17.299211+0000","last_undegraded":"2024-09-13T04:59:17.299211+0000","last_fullsized":"2024-09-13T04:59:17.299211+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":46629385,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":33,"num_read":1925,"num_read_kb":7115,"num_write":706,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"74'317","reported_seq":3934,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.091081+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:59:16.091081+0000","last_peered":"2024-09-13T04:59:16.091081+0000","last_clean":"2024-09-13T04:59:16.091081+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:59:16.091081+0000","last_undegraded":"2024-09-13T04:59:16.091081+0000","last_fullsized":"2024-09-13T04:59:16.091081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":36812324,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":5286,"num_read_kb":33748,"num_write":490,"num_write_kb":12841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"75'197","reported_seq":667,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.062410+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:59:16.062410+0000","last_peered":"2024-09-13T04:59:16.062410+0000","last_clean":"2024-09-13T04:59:16.062410+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:59:16.062410+0000","last_undegraded":"2024-09-13T04:59:16.062410+0000","last_fullsized":"2024-09-13T04:59:16.062410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":18593565,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":21,"num_read":773,"num_read_kb":6573,"num_write":181,"num_write_kb":3323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"79'683","reported_seq":5818,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.646983+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:59:17.646983+0000","last_peered":"2024-09-13T04:59:17.646983+0000","last_clean":"2024-09-13T04:59:17.646983+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:59:17.646983+0000","last_undegraded":"2024-09-13T04:59:17.646983+0000","last_fullsized":"2024-09-13T04:59:17.646983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":23509628,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":33,"num_read":7592,"num_read_kb":21669,"num_write":1009,"num_write_kb":12604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"79'283","reported_seq":4501,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.527489+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:59:17.527489+0000","last_peered":"2024-09-13T04:59:17.527489+0000","last_clean":"2024-09-13T04:59:17.527489+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:59:17.527489+0000","last_undegraded":"2024-09-13T04:59:17.527489+0000","last_fullsized":"2024-09-13T04:59:17.527489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":37645132,"num_objects":73,"num_object_clones":19,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":38,"num_read":4960,"num_read_kb":10511,"num_write":281,"num_write_kb":3309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"75'294","reported_seq":4722,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.090884+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:59:16.090884+0000","last_peered":"2024-09-13T04:59:16.090884+0000","last_clean":"2024-09-13T04:59:16.090884+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:59:16.090884+0000","last_undegraded":"2024-09-13T04:59:16.090884+0000","last_fullsized":"2024-09-13T04:59:16.090884+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":55813768,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":5174,"num_read_kb":31153,"num_write":305,"num_write_kb":16052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"76'227","reported_seq":2122,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T04:59:16.064216+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:59:16.064216+0000","last_peered":"2024-09-13T04:59:16.064216+0000","last_clean":"2024-09-13T04:59:16.064216+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:59:16.064216+0000","last_undegraded":"2024-09-13T04:59:16.064216+0000","last_fullsized":"2024-09-13T04:59:16.064216+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":18466177,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":19,"num_read":2003,"num_read_kb":11259,"num_write":242,"num_write_kb":6350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"79'481","reported_seq":1357,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T04:59:17.597011+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:59:17.597011+0000","last_peered":"2024-09-13T04:59:17.597011+0000","last_clean":"2024-09-13T04:59:17.597011+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:59:17.597011+0000","last_undegraded":"2024-09-13T04:59:17.597011+0000","last_fullsized":"2024-09-13T04:59:17.597011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":16758913,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":2853,"num_read_kb":8923,"num_write":658,"num_write_kb":4712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1006793677,"num_objects":1769,"num_object_clones":417,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":828,"num_read":120532,"num_read_kb":565341,"num_write":14106,"num_write_kb":270641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":52068352,"data_stored":413050441,"data_compressed":16161466,"data_compressed_allocated":40534016,"data_compressed_original":404809876,"omap_allocated":0,"internal_metadata":0},"log_size":11379,"ondisk_log_size":11379,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38992,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332848,"statfs":{"total":96636764160,"available":96596836352,"internally_reserved":0,"allocated":3866624,"data_stored":20448781,"data_compressed":1003423,"data_compressed_allocated":2183168,"data_compressed_original":19537643,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.747},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.95899999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.86299999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.05}]}]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46280,"kb_used_data":6972,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325560,"statfs":{"total":96636764160,"available":96589373440,"internally_reserved":0,"allocated":7139328,"data_stored":53659887,"data_compressed":1580821,"data_compressed_allocated":4751360,"data_compressed_original":52204328,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.83799999999999997}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.87},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":1.089},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.628}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.016},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.92400000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46520,"kb_used_data":7216,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325320,"statfs":{"total":96636764160,"available":96589127680,"internally_reserved":0,"allocated":7389184,"data_stored":53246929,"data_compressed":2004862,"data_compressed_allocated":5341184,"data_compressed_original":52053963,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.749}]},{"osd":1,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.68000000000000005}]},{"osd":2,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.86199999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":11.266,"5min":11.266,"15min":11.266},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.436},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.57199999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36520,"kb_used_data":9496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335320,"statfs":{"total":96636764160,"available":96599367680,"internally_reserved":0,"allocated":9723904,"data_stored":68404082,"data_compressed":2795877,"data_compressed_allocated":6873088,"data_compressed_original":66573031,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.25},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.30499999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.33900000000000002}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.629},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.876}]}]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34212,"kb_used_data":7192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337628,"statfs":{"total":96636764160,"available":96601731072,"internally_reserved":0,"allocated":7364608,"data_stored":57228471,"data_compressed":2155328,"data_compressed_allocated":5103616,"data_compressed_original":55888312,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.434},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.374}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.496}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.746}]}]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36508,"kb_used_data":9492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":9719808,"data_stored":68773774,"data_compressed":2460604,"data_compressed_allocated":7094272,"data_compressed_original":67099820,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.028}]},{"osd":2,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84199999999999997}]},{"osd":6,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.121},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.97199999999999998}]},{"osd":7,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.0649999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34332,"kb_used_data":7316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337508,"statfs":{"total":96636764160,"available":96601608192,"internally_reserved":0,"allocated":7491584,"data_stored":49597562,"data_compressed":2084970,"data_compressed_allocated":4861952,"data_compressed_original":47946012,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.21199999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":15.755000000000001,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":15.802,"5min":15.802,"15min":15.802},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1,"5min":1,"15min":1},"last":2.915},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":1.794,"15min":1.794},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":15.859999999999999,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":3.109}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":11.474,"5min":11.474,"15min":11.474},"last":3.1659999999999999},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":11.528,"5min":11.528,"15min":11.528},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.897,"5min":1.897,"15min":1.897},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":15.885999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":15.834,"5min":15.834,"15min":15.834},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":3.0670000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46108,"kb_used_data":6800,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94325732,"statfs":{"total":96636764160,"available":96589549568,"internally_reserved":0,"allocated":6963200,"data_stored":46291735,"data_compressed":2196272,"data_compressed_allocated":4517888,"data_compressed_original":44768335,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.627},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.633,"5min":4.633,"15min":4.633},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.753},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.6989999999999998,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.0660000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.008},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.84299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6557696,"data_stored":49211526,"data_compressed":2066120,"data_compressed_allocated":4845568,"data_compressed_original":47974684,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8667136,"data_stored":67676505,"data_compressed":2400118,"data_compressed_allocated":7000064,"data_compressed_original":66444460,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6443008,"data_stored":56813178,"data_compressed":2155328,"data_compressed_allocated":5103616,"data_compressed_original":55888312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8736768,"data_stored":67878381,"data_compressed":2760531,"data_compressed_allocated":6823936,"data_compressed_original":66462439,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6467584,"data_stored":52831636,"data_compressed":2004862,"data_compressed_allocated":5341184,"data_compressed_original":52053963,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6000640,"data_stored":45338084,"data_compressed":2189939,"data_compressed_allocated":4481024,"data_compressed_original":44231759,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6209536,"data_stored":53206320,"data_compressed":1580752,"data_compressed_allocated":4747264,"data_compressed_original":52167464,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2985984,"data_stored":20094811,"data_compressed":1003816,"data_compressed_allocated":2191360,"data_compressed_original":19586795,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:59:21.862 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T04:59:21.862 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:21.863 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T04:59:21.863 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T04:59:21.863 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T04:59:21.863 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T04:59:21.887 INFO:teuthology.orchestra.run.smithi001.stdout:5107 write 0x82a4c51 thru 0x82a7d61 (0x3111 bytes) 2024-09-13T04:59:21.889 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:21.889 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:21.890 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:21.890 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:21.890 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:21.894 INFO:teuthology.orchestra.run.smithi001.stdout:5109 write 0xe217fa2 thru 0xe224f86 (0xcfe5 bytes) 2024-09-13T04:59:21.902 INFO:teuthology.orchestra.run.smithi001.stdout:5110 write 0x157223f thru 0x15730dc (0xe9e bytes) 2024-09-13T04:59:21.905 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:21.906 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:21.906 INFO:teuthology.orchestra.run.smithi001.stdout:5113 write 0x3d707ed thru 0x3d719d7 (0x11eb bytes) 2024-09-13T04:59:21.910 INFO:teuthology.orchestra.run.smithi001.stdout:5114 punch from 0xdf3e0fc to 0xdf4434d, (0x6251 bytes) 2024-09-13T04:59:21.910 INFO:teuthology.orchestra.run.smithi001.stdout:5115 write 0xe81cd90 thru 0xe82a519 (0xd78a bytes) 2024-09-13T04:59:21.924 INFO:teuthology.orchestra.run.smithi001.stdout:5116 trunc from 0xea94ae8 to 0x9d78658 2024-09-13T04:59:21.947 INFO:teuthology.orchestra.run.smithi001.stdout:5118 trunc from 0x9d78658 to 0x943ae9b 2024-09-13T04:59:21.948 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:21.946+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:21.958 INFO:teuthology.orchestra.run.smithi001.stdout:5120 clone 32 order 22 su 524288 sc 10 2024-09-13T04:59:21.990 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:21.990 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "" 2024-09-13T04:59:21.990 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:22.048 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:22.046+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:22.069 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:22.069 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:22.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.032277131000001, 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:59:22.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:59:22.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:22.076 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.032231494000001, 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.077 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:22.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:59:22.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:59:22.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:59:22.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:59:22.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:59:22.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.082 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.032183410000002, 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.083 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:22.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:59:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:59:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:59:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.032087328000003, 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:59:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:59:22.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:59:22.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:59:22.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:59:22.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:59:22.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.032035957000005, 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:59:22.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:22.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.031989902999996, 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:22.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.105 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 66.031939842, 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:59:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:59:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.914524651999997, 2024-09-13T04:59:22.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:59:22.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:59:22.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:59:22.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.116 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.823096268999997, 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:59:22.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:59:22.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.823071923999997, 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.121 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:59:22.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.125 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822990570999998, 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:59:22.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822921041000001, 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.130 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:22.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:59:22.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822893765000003, 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:59:22.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822795319999997, 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.139 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:22.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.143 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822725753, 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:59:22.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822655279000003, 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.148 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:59:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:59:22.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:22.152 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822629593999999, 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:59:22.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822594406, 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.157 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:59:22.158 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.159 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.160 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822569889999997, 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.161 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.162 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.163 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.164 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.165 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 60.822500552999998, 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:22.166 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.167 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:59:22.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:59:22.169 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:59:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.170 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.170 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:22.172 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:59:22.172 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:22.173 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:22.167+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:22.235 INFO:teuthology.orchestra.run.smithi001.stdout:flattening image image_client.0-clone31 2024-09-13T04:59:22.272 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:22.270+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:22.353 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:22.353 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:22.354 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:22.354 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:22.368 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:22.409 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:22.406+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:22.473 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:22.471+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:22.477 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-13T04:59:22.504 INFO:teuthology.orchestra.run.smithi001.stdout:5121 write 0x8af2d16 thru 0x8af4d6e (0x2059 bytes) 2024-09-13T04:59:22.519 INFO:teuthology.orchestra.run.smithi001.stdout:5122 punch from 0x2a57acb to 0x2a61527, (0x9a5c bytes) 2024-09-13T04:59:22.519 INFO:teuthology.orchestra.run.smithi001.stdout:5123 write 0x1d6d684 thru 0x1d7ad35 (0xd6b2 bytes) 2024-09-13T04:59:22.555 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:22.555 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:22.555 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:22.555 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:22.555 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:22.557 INFO:teuthology.orchestra.run.smithi001.stdout:5124 writesame 0xb1bbb7c thru 0xb1cac99 data_size 0x7(0xf11e bytes) 2024-09-13T04:59:22.562 INFO:teuthology.orchestra.run.smithi001.stdout:5125 trunc from 0xb1cac9a to 0x99c71fd 2024-09-13T04:59:22.569 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:22.598 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:22.610 INFO:teuthology.orchestra.run.smithi001.stdout:5128 write 0x9e36770 thru 0x9e44b8f (0xe420 bytes) 2024-09-13T04:59:22.613 INFO:teuthology.orchestra.run.smithi001.stdout:5132 writesame 0x142f5e2 thru 0x143b9ed data_size 0x2(0xc40c bytes) 2024-09-13T04:59:22.616 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:22.614+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:22.671 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:22.671 INFO:teuthology.orchestra.run.smithi001.stdout:5134 read 0x4fe584e thru 0x4ff3b09 (0xe2bc bytes) 2024-09-13T04:59:22.673 INFO:teuthology.orchestra.run.smithi001.stdout:5135 read 0x4d878aa thru 0x4d8fca4 (0x83fb bytes) 2024-09-13T04:59:22.673 INFO:teuthology.orchestra.run.smithi001.stdout:5136 read 0x259e3aa thru 0x25a082a (0x2481 bytes) 2024-09-13T04:59:22.673 INFO:teuthology.orchestra.run.smithi001.stdout:5137 write 0x875ef3a thru 0x876ee33 (0xfefa bytes) 2024-09-13T04:59:22.675 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:22.671+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:22.684 INFO:teuthology.orchestra.run.smithi001.stdout:5138 write 0x5bf1680 thru 0x5bff67d (0xdffe bytes) 2024-09-13T04:59:22.700 INFO:teuthology.orchestra.run.smithi001.stdout:5141 writesame 0x6642560 thru 0x664d346 data_size 0x1(0xade7 bytes) 2024-09-13T04:59:22.701 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.898270105000002, 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.898235851000003, 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.89820495, 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.898153069000003, 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.718 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.898121265999997, 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.719 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:22.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.723 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.898087312999998, 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:22.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:22.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.898039576999999, 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.898000015000001, 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:22.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897952320999998, 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:22.737 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:22.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897889237999998, 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897852804000003, 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.746 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:22.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897818880999999, 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:22.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897750668, 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:22.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.757 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.759 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897716961999997, 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:22.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897666770000001, 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.764 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:22.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897360737, 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.768 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:22.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:22.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.772 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897311535, 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897263420999998, 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.777 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.781 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897211943999999, 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:22.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:22.785 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 58.897176862999999, 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:22.786 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:22.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:22.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:22.790 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:22.791 INFO:teuthology.orchestra.run.smithi001.stdout:5142 read 0x7616438 thru 0x761b64d (0x5216 bytes) 2024-09-13T04:59:22.792 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:22.792 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:22.787+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:22.795 INFO:teuthology.orchestra.run.smithi001.stdout:5143 write 0xe2c1572 thru 0xe2c3bd5 (0x2664 bytes) 2024-09-13T04:59:22.803 INFO:teuthology.orchestra.run.smithi001.stdout:5144 read 0x50813be thru 0x508dc68 (0xc8ab bytes) 2024-09-13T04:59:22.803 INFO:teuthology.orchestra.run.smithi001.stdout:5145 write 0xa2d5898 thru 0xa2d8ee2 (0x364b bytes) 2024-09-13T04:59:22.851 INFO:teuthology.orchestra.run.smithi001.stdout:5146 trunc from 0xe2c3bd6 to 0x1bf723 2024-09-13T04:59:22.893 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:22.887+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:22.968 INFO:teuthology.orchestra.run.smithi001.stdout:5148 writesame 0x508d76a thru 0x50929c5 data_size 0x2a(0x525c bytes) 2024-09-13T04:59:22.968 INFO:teuthology.orchestra.run.smithi001.stdout:5149 read 0x249e291 thru 0x249f7e9 (0x1559 bytes) 2024-09-13T04:59:22.969 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:22.969 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:22.983 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:22.984 INFO:teuthology.orchestra.run.smithi001.stdout:5151 writesame 0x5f16e84 thru 0x5f1eb29 data_size 0x5(0x7ca6 bytes) 2024-09-13T04:59:22.984 INFO:teuthology.orchestra.run.smithi001.stdout:5152 write 0x5026834 thru 0x502f761 (0x8f2e bytes) 2024-09-13T04:59:22.992 INFO:teuthology.orchestra.run.smithi001.stdout:5153 read 0x28914da thru 0x289c6d7 (0xb1fe bytes) 2024-09-13T04:59:22.992 INFO:teuthology.orchestra.run.smithi001.stdout:5154 read 0x33e0827 thru 0x33e58f9 (0x50d3 bytes) 2024-09-13T04:59:22.992 INFO:teuthology.orchestra.run.smithi001.stdout:5155 read 0x458a7b5 thru 0x458f2d6 (0x4b22 bytes) 2024-09-13T04:59:22.992 INFO:teuthology.orchestra.run.smithi001.stdout:5156 write 0x3041aef thru 0x304a9b8 (0x8eca bytes) 2024-09-13T04:59:23.002 INFO:teuthology.orchestra.run.smithi001.stdout:5157 write 0x3192a7 thru 0x327b02 (0xe85c bytes) 2024-09-13T04:59:23.033 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:23.030+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:23.033 INFO:teuthology.orchestra.run.smithi001.stdout:5159 trunc from 0x5f1eb2a to 0xcaa8257 2024-09-13T04:59:23.045 INFO:teuthology.orchestra.run.smithi001.stdout:5160 writesame 0x8efbdd9 thru 0x8f049cf data_size 0x1(0x8bf7 bytes) 2024-09-13T04:59:23.094 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:23.091+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:23.098 INFO:teuthology.orchestra.run.smithi001.stdout:5161 read 0x9fd0f7f thru 0x9fd4bd9 (0x3c5b bytes) 2024-09-13T04:59:23.099 INFO:teuthology.orchestra.run.smithi001.stdout:5162 punch from 0x7c39666 to 0x7c3fe40, (0x67da bytes) 2024-09-13T04:59:23.107 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:23.107 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:23.108 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:23.119 INFO:teuthology.orchestra.run.smithi001.stdout:5163 trunc from 0xcaa8257 to 0x521dd66 2024-09-13T04:59:23.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:23.154 INFO:teuthology.orchestra.run.smithi001.stdout:5164 trunc from 0x521dd66 to 0x23dbfed 2024-09-13T04:59:23.181 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:23.182 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:23.192 INFO:teuthology.orchestra.run.smithi001.stdout:5166 trunc from 0x23dbfed to 0x6004617 2024-09-13T04:59:23.212 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:23.210+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:23.223 INFO:teuthology.orchestra.run.smithi001.stdout:5168 write 0xd3652fc thru 0xd365dd6 (0xadb bytes) 2024-09-13T04:59:23.228 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:23.228 INFO:teuthology.orchestra.run.smithi001.stdout:5170 punch from 0xb4003f6 to 0xb40e7f4, (0xe3fe bytes) 2024-09-13T04:59:23.242 INFO:teuthology.orchestra.run.smithi001.stdout:5172 trunc from 0xd365dd7 to 0x50ef8b3 2024-09-13T04:59:23.258 INFO:teuthology.orchestra.run.smithi001.stdout:5173 write 0x38b887a thru 0x38c1026 (0x87ad bytes) 2024-09-13T04:59:23.264 INFO:teuthology.orchestra.run.smithi001.stdout:5175 read 0x47cf5a6 thru 0x47d22c1 (0x2d1c bytes) 2024-09-13T04:59:23.265 INFO:teuthology.orchestra.run.smithi001.stdout:5176 read 0x14e2db8 thru 0x14e8c61 (0x5eaa bytes) 2024-09-13T04:59:23.265 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:23.280 INFO:teuthology.orchestra.run.smithi001.stdout:5178 writesame 0x91f5cee thru 0x91fd6a1 data_size 0x2(0x79b4 bytes) 2024-09-13T04:59:23.280 INFO:teuthology.orchestra.run.smithi001.stdout:5180 writesame 0x6b117d2 thru 0x6b20eed data_size 0xa(0xf71c bytes) 2024-09-13T04:59:23.293 INFO:teuthology.orchestra.run.smithi001.stdout:5181 write 0x2b33204 thru 0x2b3cf5f (0x9d5c bytes) 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49885 3.b 3:d43de71f:::rbd_data.112a40bf6452.0000000000000003:f [sparse-read 0~1843200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.264146709999999, 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015118388, 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49885 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028150+0000", 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028157+0000", 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5510000000000002e-06 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028174+0000", 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6631000000000001e-05 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028183+0000", 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6100000000000006e-06 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028308+0000", 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012532600000000001 2024-09-13T04:59:23.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028557+0000", 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024922200000000001 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.043270+0000", 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014712578 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.298 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49972 3.b 3:d2a26831:::rbd_data.112ae3a54590.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.112881303999998, 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021756569999999999, 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49972 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179414+0000", 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:23.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180536+0000", 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001122381 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180563+0000", 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6211999999999999e-05 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180582+0000", 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9630999999999999e-05 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180662+0000", 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9411000000000003e-05 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180947+0000", 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000285166 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181294+0000", 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034697800000000001 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200928+0000", 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019634766000000001 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201089+0000", 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016092299999999999 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201142+0000", 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2855000000000001e-05 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201173+0000", 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1096999999999997e-05 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49973 3.b 3:d63ae98e:::rbd_data.112ae3a54590.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:23.304 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.111687764999999, 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026409885000000001, 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49973 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:23.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180608+0000", 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181851+0000", 2024-09-13T04:59:23.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012428630000000001 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181894+0000", 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2710000000000003e-05 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181909+0000", 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5092999999999999e-05 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.186892+0000", 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049824229999999997 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187123+0000", 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023144999999999999 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187416+0000", 2024-09-13T04:59:23.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029303400000000001 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200934+0000", 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013517714 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206897+0000", 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059634880000000003 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206972+0000", 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5165000000000003e-05 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207020+0000", 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7821000000000002e-05 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49975 3.b 3:d71a7fd7:::rbd_data.112ae3a54590.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1826816~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:23.310 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 47.110029005000001, 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025198755, 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49975 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182267+0000", 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183414+0000", 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011469270000000001 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183442+0000", 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8189000000000001e-05 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183458+0000", 2024-09-13T04:59:23.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6073999999999999e-05 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.195878+0000", 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01241922 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196039+0000", 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161058 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:23.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196279+0000", 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240676 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201295+0000", 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0050153999999999997 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207382+0000", 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0060871650000000003 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207432+0000", 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9882000000000001e-05 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207468+0000", 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6140000000000003e-05 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.316 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79481 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.840963096999999, 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010622207, 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79481 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451331+0000", 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.318 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451350+0000", 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8448999999999999e-05 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451425+0000", 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4929e-05 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451448+0000", 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3618999999999999e-05 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451655+0000", 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020621600000000001 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451671+0000", 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6013000000000001e-05 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461479+0000", 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098082340000000007 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461702+0000", 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022309900000000001 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461957+0000", 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000255171 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79525 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.321 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.840808890000002, 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010933693, 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79525 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451488+0000", 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451497+0000", 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8359999999999998e-06 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451527+0000", 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0015999999999999e-05 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451543+0000", 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6059999999999999e-05 2024-09-13T04:59:23.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451797+0000", 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025463100000000001 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451810+0000", 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2999999999999999e-05 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462017+0000", 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010207101999999999 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462210+0000", 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192649 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462423+0000", 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021292899999999999 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79695 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.837104803999999, 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010472398000000001, 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79695 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455192+0000", 2024-09-13T04:59:23.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455198+0000", 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9399999999999999e-06 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455215+0000", 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.632e-05 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455229+0000", 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4146e-05 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455854+0000", 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00062516999999999998 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:23.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455860+0000", 2024-09-13T04:59:23.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7300000000000002e-06 2024-09-13T04:59:23.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.464832+0000", 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0089722559999999996 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465369+0000", 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00053748100000000003 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465666+0000", 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029624799999999999 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79696 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.837057411, 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011064011, 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79696 2024-09-13T04:59:23.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455240+0000", 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455246+0000", 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5830000000000003e-06 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455264+0000", 2024-09-13T04:59:23.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7450000000000001e-05 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455277+0000", 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3016e-05 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455886+0000", 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000609637 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455892+0000", 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5199999999999997e-06 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465781+0000", 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098892089999999995 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466021+0000", 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024036500000000001 2024-09-13T04:59:23.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466305+0000", 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028331100000000001 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79718 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.837009549000001, 2024-09-13T04:59:23.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011634479, 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79718 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455288+0000", 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455293+0000", 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455310+0000", 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6393e-05 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455323+0000", 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3013000000000001e-05 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455918+0000", 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059522399999999999 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455923+0000", 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4060000000000004e-06 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466385+0000", 2024-09-13T04:59:23.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010462154 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466593+0000", 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020723900000000001 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466923+0000", 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00033036300000000002 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.343 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79737 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.836964222999999, 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011808300000000001, 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79737 2024-09-13T04:59:23.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455333+0000", 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455339+0000", 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7570000000000001e-06 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455355+0000", 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.632e-05 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455368+0000", 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2588999999999999e-05 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455948+0000", 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058065099999999995 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455954+0000", 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466957+0000", 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011003361 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467033+0000", 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5644999999999998e-05 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467142+0000", 2024-09-13T04:59:23.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109527 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79757 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 393216~38443] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.836918911000001, 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013373497999999999, 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79757 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455378+0000", 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455384+0000", 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6200000000000004e-06 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455400+0000", 2024-09-13T04:59:23.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5973000000000001e-05 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455412+0000", 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2689e-05 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455979+0000", 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00056651800000000001 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455984+0000", 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.49e-06 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467237+0000", 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011252376999999999 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467453+0000", 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000216338 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468753+0000", 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012995459999999999 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79776 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.836874065, 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013932595000000001, 2024-09-13T04:59:23.354 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79776 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455423+0000", 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455429+0000", 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455446+0000", 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6877000000000001e-05 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455459+0000", 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2866e-05 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456009+0000", 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000550838 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456015+0000", 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6060000000000002e-06 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468961+0000", 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012946301 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469181+0000", 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000220075 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469357+0000", 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000175279 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:79796 3.0 e60/24 3:003e46ae:::rbd_data.112acc4fa3fd.0000000000000000:head v 60'366, mlcod=58'364)", 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.801541388, 2024-09-13T04:59:23.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025297777, 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490755+0000", 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494145+0000", 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033899300000000002 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494187+0000", 2024-09-13T04:59:23.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1950999999999999e-05 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494201+0000", 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4134000000000001e-05 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494267+0000", 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5815999999999995e-05 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494296+0000", 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8597000000000001e-05 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516017+0000", 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021721285 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516054+0000", 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7389999999999999e-05 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79853 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 28.748561202000001, 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043236985999999998, 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.364 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79853 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543734+0000", 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548313+0000", 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045782449999999999 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548339+0000", 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6948999999999999e-05 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548356+0000", 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6160000000000001e-05 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548459+0000", 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000103453 2024-09-13T04:59:23.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548718+0000", 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025891599999999999 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.549442+0000", 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00072383699999999996 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586544+0000", 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037102028000000002 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586859+0000", 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000315237 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586924+0000", 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5315999999999996e-05 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586974+0000", 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9481000000000003e-05 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81749 3.b 3:d52afe2d:::rbd_data.112a86dc1e25.000000000000000d:head [stat,delete] snapc 18=[18] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.220094112999998, 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010905319, 2024-09-13T04:59:23.370 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81749 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072201+0000", 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072215+0000", 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3859000000000001e-05 2024-09-13T04:59:23.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072244+0000", 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8612999999999999e-05 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072265+0000", 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0893000000000001e-05 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.073443+0000", 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011780359999999999 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.073753+0000", 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031037700000000003 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.074309+0000", 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00055601200000000004 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.076960+0000", 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002650856 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.082880+0000", 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005919901 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.083013+0000", 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000133168 2024-09-13T04:59:23.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.083109+0000", 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6163999999999996e-05 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81735 3.8 e64/24 3:148b359f:::rbd_data.112a86dc1e25.0000000000000006:head v 64'139, mlcod=64'137)", 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.214577893000001, 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011631831, 2024-09-13T04:59:23.376 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077718+0000", 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078104+0000", 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000385852 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078144+0000", 2024-09-13T04:59:23.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9858999999999999e-05 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078163+0000", 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8666999999999999e-05 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078282+0000", 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000119784 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078334+0000", 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1712000000000002e-05 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.089290+0000", 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010955515000000001 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.089352+0000", 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2389000000000002e-05 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.380 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81782 3.c e64/24 3:33317f5e:::rbd_data.112a86dc1e25.000000000000000b:head v 64'410, mlcod=63'408)", 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.197245995999999, 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012792019999999999, 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:23.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095051+0000", 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095635+0000", 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058456600000000001 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095660+0000", 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4374000000000001e-05 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095670+0000", 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0631999999999999e-05 2024-09-13T04:59:23.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095743+0000", 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3015000000000005e-05 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095775+0000", 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2092999999999999e-05 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107755+0000", 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011979949 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107844+0000", 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8858999999999997e-05 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81874 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [sparse-read 0~2423869] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:23.385 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 25.024580432, 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015773423000000002, 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81874 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267715+0000", 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267729+0000", 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4399999999999999e-05 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267761+0000", 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2112e-05 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267776+0000", 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4823e-05 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267903+0000", 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000127014 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.268135+0000", 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023146899999999999 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.283491+0000", 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015356491 2024-09-13T04:59:23.389 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:96142 3.0 e84/24 3:05f91f13:::rbd_data.112a487824ce.0000000000000000:head v 84'514, mlcod=82'512)", 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 1.044238977, 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013444012999999999, 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:23.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248058+0000", 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248437+0000", 2024-09-13T04:59:23.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000378945 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248455+0000", 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7791999999999998e-05 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248467+0000", 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1568e-05 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248575+0000", 2024-09-13T04:59:23.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108637 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248605+0000", 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9471999999999998e-05 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.261461+0000", 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012856627000000001 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.261503+0000", 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656000000000002e-05 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:96241 3.b 3:d219a614:::rbd_data.112a3f73279b.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3069572~54962] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.767488273, 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014882869999999999, 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:23.394 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 96241 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524806+0000", 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525284+0000", 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047714799999999999 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525320+0000", 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6219e-05 2024-09-13T04:59:23.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525337+0000", 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7183000000000001e-05 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525421+0000", 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3944999999999997e-05 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525613+0000", 2024-09-13T04:59:23.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019242099999999999 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525898+0000", 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028419399999999998 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.533236+0000", 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073389190000000002 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:23.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539592+0000", 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.006355688 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539646+0000", 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4360000000000001e-05 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539693+0000", 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6205999999999998e-05 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:23.400 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:23.400 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:23.401 INFO:teuthology.orchestra.run.smithi001.stdout:5182 writesame 0x2ff18a1 thru 0x2ff83e5 data_size 0x1(0x6b45 bytes) 2024-09-13T04:59:23.401 INFO:teuthology.orchestra.run.smithi001.stdout:5183 flatten 2024-09-13T04:59:23.401 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:23.402 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:23.402 INFO:teuthology.orchestra.run.smithi001.stdout:5184 trunc from 0x91fd6a2 to 0x15841ff 2024-09-13T04:59:23.402 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:23.403 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:23.398+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:23.403 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:23.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:23.404 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:23.404 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:23.405 INFO:teuthology.orchestra.run.smithi001.stdout:5185 write 0xe7ab012 thru 0xe7b182b (0x681a bytes) 2024-09-13T04:59:23.405 INFO:teuthology.orchestra.run.smithi001.stdout:5186 read 0xa8b6e83 thru 0xa8c3af7 (0xcc75 bytes) 2024-09-13T04:59:23.405 INFO:teuthology.orchestra.run.smithi001.stdout:5187 write 0x40492df thru 0x404dd63 (0x4a85 bytes) 2024-09-13T04:59:23.405 INFO:teuthology.orchestra.run.smithi001.stdout:5189 trunc from 0xe7b182c to 0xe2e2496 2024-09-13T04:59:23.405 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:23.405 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:23.407 INFO:teuthology.orchestra.run.smithi001.stdout:5190 punch from 0x5b6369d to 0x5b65478, (0x1ddb bytes) 2024-09-13T04:59:23.407 INFO:teuthology.orchestra.run.smithi001.stdout:5191 read 0x1f9a02a thru 0x1fa9344 (0xf31b bytes) 2024-09-13T04:59:23.407 INFO:teuthology.orchestra.run.smithi001.stdout:5192 write 0x386b55d thru 0x386cdf5 (0x1899 bytes) 2024-09-13T04:59:23.412 INFO:teuthology.orchestra.run.smithi001.stdout:5193 write 0xbfb68cd thru 0xbfc2f81 (0xc6b5 bytes) 2024-09-13T04:59:23.422 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:23.422 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:23.424 INFO:teuthology.orchestra.run.smithi001.stdout:5195 write 0x9115976 thru 0x9116f2b (0x15b6 bytes) 2024-09-13T04:59:23.429 INFO:teuthology.orchestra.run.smithi001.stdout:5196 write 0x3b01553 thru 0x3b03384 (0x1e32 bytes) 2024-09-13T04:59:23.432 INFO:teuthology.orchestra.run.smithi001.stdout:5197 read 0xb8a6b15 thru 0xb8ac784 (0x5c70 bytes) 2024-09-13T04:59:23.432 INFO:teuthology.orchestra.run.smithi001.stdout:5198 write 0xa75c886 thru 0xa75f69d (0x2e18 bytes) 2024-09-13T04:59:23.438 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:23.438 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:23.438 INFO:teuthology.orchestra.run.smithi001.stdout:5201 write 0xb7434ee thru 0xb7489f2 (0x5505 bytes) 2024-09-13T04:59:23.443 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:23.443 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:23.445 INFO:teuthology.orchestra.run.smithi001.stdout:5202 write 0x58bc4d3 thru 0x58ca2fd (0xde2b bytes) 2024-09-13T04:59:23.456 INFO:teuthology.orchestra.run.smithi001.stdout:5203 write 0x772df73 thru 0x7734b52 (0x6be0 bytes) 2024-09-13T04:59:23.462 INFO:teuthology.orchestra.run.smithi001.stdout:5204 read 0x487685e thru 0x487f18a (0x892d bytes) 2024-09-13T04:59:23.463 INFO:teuthology.orchestra.run.smithi001.stdout:5205 read 0xc22cf61 thru 0xc2385f9 (0xb699 bytes) 2024-09-13T04:59:23.463 INFO:teuthology.orchestra.run.smithi001.stdout:5206 writesame 0x1d52ff thru 0x1db08f data_size 0x1(0x5d91 bytes) 2024-09-13T04:59:23.463 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:23.464 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:23.479 INFO:teuthology.orchestra.run.smithi001.stdout:5207 punch from 0xc0e8641 to 0xc0f1c82, (0x9641 bytes) 2024-09-13T04:59:23.479 INFO:teuthology.orchestra.run.smithi001.stdout:5209 write 0x5b07ec6 thru 0x5b157d6 (0xd911 bytes) 2024-09-13T04:59:23.486 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:23.491 INFO:teuthology.orchestra.run.smithi001.stdout:5211 write 0x7bc8316 thru 0x7bce30a (0x5ff5 bytes) 2024-09-13T04:59:23.498 INFO:teuthology.orchestra.run.smithi001.stdout:5212 write 0x1d9864b thru 0x1d9dd8a (0x5740 bytes) 2024-09-13T04:59:23.501 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:23.499+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:23.507 INFO:teuthology.orchestra.run.smithi001.stdout:5213 read 0xb76fd94 thru 0xb772ab9 (0x2d26 bytes) 2024-09-13T04:59:23.508 INFO:teuthology.orchestra.run.smithi001.stdout:5214 write 0xc5049a5 thru 0xc505af3 (0x114f bytes) 2024-09-13T04:59:23.511 INFO:teuthology.orchestra.run.smithi001.stdout:5215 write 0xd51b484 thru 0xd51f385 (0x3f02 bytes) 2024-09-13T04:59:23.515 INFO:teuthology.orchestra.run.smithi001.stdout:5216 write 0x3d59dc5 thru 0x3d5e12f (0x436b bytes) 2024-09-13T04:59:23.520 INFO:teuthology.orchestra.run.smithi001.stdout:5218 writesame 0x66d501e thru 0x66e1f92 data_size 0x3f(0xcf75 bytes) 2024-09-13T04:59:23.530 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:23.530 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:23.545 INFO:teuthology.orchestra.run.smithi001.stdout:5219 trunc from 0xe2e2496 to 0x165d4eb 2024-09-13T04:59:23.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:23.588 INFO:teuthology.orchestra.run.smithi001.stdout:5224 writesame 0xb2e2d9c thru 0xb2f11af data_size 0x16(0xe414 bytes) 2024-09-13T04:59:23.588 INFO:teuthology.orchestra.run.smithi001.stdout:5225 write 0x56f2bdc thru 0x56f8aba (0x5edf bytes) 2024-09-13T04:59:23.596 INFO:teuthology.orchestra.run.smithi001.stdout:5226 punch from 0xa4bb093 to 0xa4c247b, (0x73e8 bytes) 2024-09-13T04:59:23.596 INFO:teuthology.orchestra.run.smithi001.stdout:5227 read 0x31c8a5c thru 0x31cb39d (0x2942 bytes) 2024-09-13T04:59:23.596 INFO:teuthology.orchestra.run.smithi001.stdout:5229 write 0x4b0320d thru 0x4b13020 (0xfe14 bytes) 2024-09-13T04:59:23.602 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:23.598+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:23.608 INFO:teuthology.orchestra.run.smithi001.stdout:5230 writesame 0x6a81fb0 thru 0x6a86f58 data_size 0x1(0x4fa9 bytes) 2024-09-13T04:59:23.629 INFO:teuthology.orchestra.run.smithi001.stdout:5231 writesame 0x11b3e69 thru 0x11bbd15 data_size 0x1(0x7ead bytes) 2024-09-13T04:59:23.657 INFO:teuthology.orchestra.run.smithi001.stdout:5232 read 0x4caee49 thru 0x4cbd63c (0xe7f4 bytes) 2024-09-13T04:59:23.658 INFO:teuthology.orchestra.run.smithi001.stdout:5233 writesame 0x9dab9c4 thru 0x9dbb50a data_size 0x1(0xfb47 bytes) 2024-09-13T04:59:23.703 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:23.699+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:23.711 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:23.711 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:23.721 INFO:teuthology.orchestra.run.smithi001.stdout:5235 read 0x1f63dd0 thru 0x1f66a01 (0x2c32 bytes) 2024-09-13T04:59:23.724 INFO:teuthology.orchestra.run.smithi001.stdout:5236 read 0x37064f1 thru 0x3715cc7 (0xf7d7 bytes) 2024-09-13T04:59:23.727 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:23.727 INFO:teuthology.orchestra.run.smithi001.stdout:5237 punch from 0x49a38c1 to 0x49af42f, (0xbb6e bytes) 2024-09-13T04:59:23.750 INFO:teuthology.orchestra.run.smithi001.stdout:5240 trunc from 0xb2f11b0 to 0x130d79a 2024-09-13T04:59:23.782 INFO:teuthology.orchestra.run.smithi001.stdout:5241 punch from 0x4e89e4 to 0x4f387f, (0xae9b bytes) 2024-09-13T04:59:23.804 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:23.799+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:23.815 INFO:teuthology.orchestra.run.smithi001.stdout:5242 write 0xab2097e thru 0xab305a3 (0xfc26 bytes) 2024-09-13T04:59:23.828 INFO:teuthology.orchestra.run.smithi001.stdout:5244 trunc from 0xab305a4 to 0x85f5839 2024-09-13T04:59:23.868 INFO:teuthology.orchestra.run.smithi001.stdout:5245 trunc from 0x85f5839 to 0x3667fc 2024-09-13T04:59:23.878 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:23.879 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:23.892 INFO:teuthology.orchestra.run.smithi001.stdout:5246 punch from 0x2c22a to 0x3918d, (0xcf63 bytes) 2024-09-13T04:59:23.894 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:23.905 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:23.902+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:5248 write 0x3d0e3ce thru 0x3d14a52 (0x6685 bytes) 2024-09-13T04:59:23.919 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:23.928 INFO:teuthology.orchestra.run.smithi001.stdout:5251 trunc from 0x3d14a53 to 0x8bf2d87 2024-09-13T04:59:23.941 INFO:teuthology.orchestra.run.smithi001.stdout:5253 trunc from 0x8bf2d87 to 0x6613582 2024-09-13T04:59:23.952 INFO:teuthology.orchestra.run.smithi001.stdout:5254 trunc from 0x6613582 to 0x87ae162 2024-09-13T04:59:23.961 INFO:teuthology.orchestra.run.smithi001.stdout:5255 read 0xc1ab09 thru 0xc1c65d (0x1b55 bytes) 2024-09-13T04:59:23.961 INFO:teuthology.orchestra.run.smithi001.stdout:5256 write 0x7f886db thru 0x7f8dd59 (0x567f bytes) 2024-09-13T04:59:23.969 INFO:teuthology.orchestra.run.smithi001.stdout:5257 punch from 0x150a086 to 0x151631d, (0xc297 bytes) 2024-09-13T04:59:23.969 INFO:teuthology.orchestra.run.smithi001.stdout:5260 writesame 0x3665ae5 thru 0x36667b1 data_size 0x1(0xccd bytes) 2024-09-13T04:59:23.977 INFO:teuthology.orchestra.run.smithi001.stdout:5262 punch from 0x15a1ac8 to 0x15ac799, (0xacd1 bytes) 2024-09-13T04:59:23.977 INFO:teuthology.orchestra.run.smithi001.stdout:5263 clone 33 order 21 su 524288 sc 10 2024-09-13T04:59:24.006 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:24.002+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:24.036 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:24.036 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:24.051 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:24.106 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:24.102+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:24.174 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:59:24.207 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:24.202+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:24.233 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:24.233 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:24.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:24.308 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:24.306+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:24.365 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-13T04:59:24.383 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:24.383 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:24.402 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:24.450 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:24.446+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:24.509 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:24.506+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:24.532 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:24.532 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:24.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:24.611 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:24.610+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:24.654 INFO:teuthology.orchestra.run.smithi001.stdout:5265 trunc from 0x87ae162 to 0xd7f5334 2024-09-13T04:59:24.672 INFO:teuthology.orchestra.run.smithi001.stdout:5266 read 0xd053fad thru 0xd063da7 (0xfdfb bytes) 2024-09-13T04:59:24.672 INFO:teuthology.orchestra.run.smithi001.stdout:5267 punch from 0x718be67 to 0x71903d7, (0x4570 bytes) 2024-09-13T04:59:24.672 INFO:teuthology.orchestra.run.smithi001.stdout:5268 write 0x92143c9 thru 0x921b9e8 (0x7620 bytes) 2024-09-13T04:59:24.680 INFO:teuthology.orchestra.run.smithi001.stdout:5269 writesame 0xbe33c88 thru 0xbe35c6e data_size 0x1(0x1fe7 bytes) 2024-09-13T04:59:24.690 INFO:teuthology.orchestra.run.smithi001.stdout:5270 writesame 0x3d4bd47 thru 0x3d59ca1 data_size 0x1(0xdf5b bytes) 2024-09-13T04:59:24.697 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:24.697 INFO:teuthology.orchestra.run.smithi001.stdout:5272 writesame 0x4667dc2 thru 0x466e26a data_size 0x1(0x64a9 bytes) 2024-09-13T04:59:24.711 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:24.706+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:24.723 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:24.723 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:24.738 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:24.739 INFO:teuthology.orchestra.run.smithi001.stdout:5273 read 0xa06465a thru 0xa06d61e (0x8fc5 bytes) 2024-09-13T04:59:24.755 INFO:teuthology.orchestra.run.smithi001.stdout:5274 trunc from 0xd7f5334 to 0x447a0ed 2024-09-13T04:59:24.788 INFO:teuthology.orchestra.run.smithi001.stdout:5275 read 0xb0669 thru 0xbcc96 (0xc62e bytes) 2024-09-13T04:59:24.790 INFO:teuthology.orchestra.run.smithi001.stdout:5276 writesame 0x3ad9773 thru 0x3ad9eb4 data_size 0x1(0x742 bytes) 2024-09-13T04:59:24.797 INFO:teuthology.orchestra.run.smithi001.stdout:5278 write 0xa04086 thru 0xa0d2bd (0x9238 bytes) 2024-09-13T04:59:24.812 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:24.806+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:24.828 INFO:teuthology.orchestra.run.smithi001.stdout:5279 write 0x2f1733 thru 0x2fc778 (0xb046 bytes) 2024-09-13T04:59:24.840 INFO:teuthology.orchestra.run.smithi001.stdout:5280 write 0x1bb0fea thru 0x1bb1f08 (0xf1f bytes) 2024-09-13T04:59:24.849 INFO:teuthology.orchestra.run.smithi001.stdout:5281 punch from 0x2c45752 to 0x2c4eaa1, (0x934f bytes) 2024-09-13T04:59:24.873 INFO:teuthology.orchestra.run.smithi001.stdout:5282 write 0xb79d636 thru 0xb7a1241 (0x3c0c bytes) 2024-09-13T04:59:24.876 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:24.876 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:24.881 INFO:teuthology.orchestra.run.smithi001.stdout:5283 write 0x31d26b7 thru 0x31d28a6 (0x1f0 bytes) 2024-09-13T04:59:24.888 INFO:teuthology.orchestra.run.smithi001.stdout:5284 punch from 0xb36ac57 to 0xb3798db, (0xec84 bytes) 2024-09-13T04:59:24.888 INFO:teuthology.orchestra.run.smithi001.stdout:5286 writesame 0x8972453 thru 0x897c577 data_size 0x1(0xa125 bytes) 2024-09-13T04:59:24.892 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:24.913 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:24.910+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:24.959 INFO:teuthology.orchestra.run.smithi001.stdout:5287 write 0xe067a89 thru 0xe075f89 (0xe501 bytes) 2024-09-13T04:59:24.964 INFO:teuthology.orchestra.run.smithi001.stdout:5289 write 0xd634164 thru 0xd6367ef (0x268c bytes) 2024-09-13T04:59:24.969 INFO:teuthology.orchestra.run.smithi001.stdout:5291 write 0x65dd6c6 thru 0x65e8c33 (0xb56e bytes) 2024-09-13T04:59:24.979 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:24.979 INFO:teuthology.orchestra.run.smithi001.stdout:5295 read 0x32045d7 thru 0x320b38e (0x6db8 bytes) 2024-09-13T04:59:24.979 INFO:teuthology.orchestra.run.smithi001.stdout:5296 read 0xab6649f thru 0xab6ff4b (0x9aad bytes) 2024-09-13T04:59:24.979 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:24.980 INFO:teuthology.orchestra.run.smithi001.stdout:5299 punch from 0x63da426 to 0x63db9f5, (0x15cf bytes) 2024-09-13T04:59:24.983 INFO:teuthology.orchestra.run.smithi001.stdout:5300 trunc from 0xe075f8a to 0xbe8d658 2024-09-13T04:59:25.004 INFO:teuthology.orchestra.run.smithi001.stdout:5301 write 0x672bd77 thru 0x6730b93 (0x4e1d bytes) 2024-09-13T04:59:25.008 INFO:teuthology.orchestra.run.smithi001.stdout:5303 punch from 0x3df3e12 to 0x3dfe729, (0xa917 bytes) 2024-09-13T04:59:25.014 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:25.010+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:25.025 INFO:teuthology.orchestra.run.smithi001.stdout:5304 trunc from 0xbe8d658 to 0x11b3371 2024-09-13T04:59:25.071 INFO:teuthology.orchestra.run.smithi001.stdout:5305 trunc from 0x11b3371 to 0x7b55a5d 2024-09-13T04:59:25.075 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:25.075 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:25.082 INFO:teuthology.orchestra.run.smithi001.stdout:5306 read 0x7246ab6 thru 0x7248dd4 (0x231f bytes) 2024-09-13T04:59:25.085 INFO:teuthology.orchestra.run.smithi001.stdout:5308 trunc from 0x7b55a5d to 0x930b5c7 2024-09-13T04:59:25.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:25.097 INFO:teuthology.orchestra.run.smithi001.stdout:5309 write 0x1f469cd thru 0x1f545c3 (0xdbf7 bytes) 2024-09-13T04:59:25.109 INFO:teuthology.orchestra.run.smithi001.stdout:5310 read 0x34e30a thru 0x34f40b (0x1102 bytes) 2024-09-13T04:59:25.110 INFO:teuthology.orchestra.run.smithi001.stdout:5311 write 0x4f29551 thru 0x4f29a36 (0x4e6 bytes) 2024-09-13T04:59:25.114 INFO:teuthology.orchestra.run.smithi001.stdout:5312 punch from 0x1168121 to 0x116ebf1, (0x6ad0 bytes) 2024-09-13T04:59:25.114 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:25.110+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:25.129 INFO:teuthology.orchestra.run.smithi001.stdout:5313 write 0xc994516 thru 0xc9970e1 (0x2bcc bytes) 2024-09-13T04:59:25.134 INFO:teuthology.orchestra.run.smithi001.stdout:5314 write 0x860571c thru 0x860d55b (0x7e40 bytes) 2024-09-13T04:59:25.140 INFO:teuthology.orchestra.run.smithi001.stdout:5316 punch from 0x20d0298 to 0x20d9c02, (0x996a bytes) 2024-09-13T04:59:25.140 INFO:teuthology.orchestra.run.smithi001.stdout:5317 write 0x4871b42 thru 0x487dd2d (0xc1ec bytes) 2024-09-13T04:59:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:59:25.152 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:25.152 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:25.152 INFO:teuthology.orchestra.run.smithi001.stdout:5322 read 0x36433fb thru 0x3647f0d (0x4b13 bytes) 2024-09-13T04:59:25.152 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:25.152 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:25.171 INFO:teuthology.orchestra.run.smithi001.stdout:5327 trunc from 0xc9970e2 to 0xe0c7a 2024-09-13T04:59:25.215 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:25.210+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:25.218 INFO:teuthology.orchestra.run.smithi001.stdout:5329 punch from 0xd2f4c to 0xe0c7a, (0xdd2e bytes) 2024-09-13T04:59:25.239 INFO:teuthology.orchestra.run.smithi001.stdout:5330 write 0x701a01a thru 0x7022b39 (0x8b20 bytes) 2024-09-13T04:59:25.244 INFO:teuthology.orchestra.run.smithi001.stdout:5332 read 0x3b2df6c thru 0x3b387e8 (0xa87d bytes) 2024-09-13T04:59:25.249 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:25.249 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:25.264 INFO:teuthology.orchestra.run.smithi001.stdout:5333 write 0xd2e59f0 thru 0xd2eb654 (0x5c65 bytes) 2024-09-13T04:59:25.267 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:25.271 INFO:teuthology.orchestra.run.smithi001.stdout:5334 read 0x2652d9a thru 0x265cc2a (0x9e91 bytes) 2024-09-13T04:59:25.271 INFO:teuthology.orchestra.run.smithi001.stdout:5335 read 0x1913aa0 thru 0x191458b (0xaec bytes) 2024-09-13T04:59:25.271 INFO:teuthology.orchestra.run.smithi001.stdout:5337 write 0x6604b1b thru 0x6605331 (0x817 bytes) 2024-09-13T04:59:25.280 INFO:teuthology.orchestra.run.smithi001.stdout:5338 trunc from 0xd2eb655 to 0x85b5955 2024-09-13T04:59:25.297 INFO:teuthology.orchestra.run.smithi001.stdout:5339 writesame 0x5ea75fa thru 0x5ea8a49 data_size 0x8(0x1450 bytes) 2024-09-13T04:59:25.302 INFO:teuthology.orchestra.run.smithi001.stdout:5341 writesame 0x4ed2e5d thru 0x4ed8c17 data_size 0x1(0x5dbb bytes) 2024-09-13T04:59:25.316 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:25.310+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:25.320 INFO:teuthology.orchestra.run.smithi001.stdout:5342 write 0x2cfc93d thru 0x2d03b53 (0x7217 bytes) 2024-09-13T04:59:25.328 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:25.328 INFO:teuthology.orchestra.run.smithi001.stdout:5346 writesame 0x7e8b812 thru 0x7e999b7 data_size 0x11(0xe1a6 bytes) 2024-09-13T04:59:25.336 INFO:teuthology.orchestra.run.smithi001.stdout:5347 write 0x43f9aaf thru 0x4409247 (0xf799 bytes) 2024-09-13T04:59:25.343 INFO:teuthology.orchestra.run.smithi001.stdout:5348 punch from 0x499525c to 0x499f36d, (0xa111 bytes) 2024-09-13T04:59:25.365 INFO:teuthology.orchestra.run.smithi001.stdout:5349 write 0xe2b38b5 thru 0xe2b740e (0x3b5a bytes) 2024-09-13T04:59:25.370 INFO:teuthology.orchestra.run.smithi001.stdout:5351 writesame 0x8a00a45 thru 0x8a0bc6f data_size 0x1(0xb22b bytes) 2024-09-13T04:59:25.399 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:25.399 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:25.400 INFO:teuthology.orchestra.run.smithi001.stdout:5352 write 0x625e845 thru 0x6269e92 (0xb64e bytes) 2024-09-13T04:59:25.411 INFO:teuthology.orchestra.run.smithi001.stdout:5353 punch from 0x5ce6dac to 0x5ce79cd, (0xc21 bytes) 2024-09-13T04:59:25.416 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:25.417 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:25.414+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:25.434 INFO:teuthology.orchestra.run.smithi001.stdout:5354 trunc from 0xe2b740f to 0x12a29c6 2024-09-13T04:59:25.458 INFO:teuthology.orchestra.run.smithi001.stdout:5355 punch from 0xde004c to 0xde7035, (0x6fe9 bytes) 2024-09-13T04:59:25.458 INFO:teuthology.orchestra.run.smithi001.stdout:5356 read 0x802d23 thru 0x8072f7 (0x45d5 bytes) 2024-09-13T04:59:25.478 INFO:teuthology.orchestra.run.smithi001.stdout:5357 write 0x6efec00 thru 0x6f06059 (0x745a bytes) 2024-09-13T04:59:25.486 INFO:teuthology.orchestra.run.smithi001.stdout:5358 clone 34 order 22 su 524288 sc 2 2024-09-13T04:59:25.506 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:25.549 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:25.549 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:25.553 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:25.550+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:25.565 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:25.619 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:25.614+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:25.656 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:25.656 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:25.672 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:25.686 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:25.686 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:25.687 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:25.688 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:25.700 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:25.705 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:25.705 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:25.723 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:25.723 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:25.741 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:25.745 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:25.742+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:25.761 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:25.779 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:25.779 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:25.820 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:25.818+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:25.872 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.011565175000001, 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T04:59:25.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.011538264000002, 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T04:59:25.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.007792627999997, 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T04:59:25.882 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T04:59:25.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T04:59:25.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.004249303000002, 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.887 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T04:59:25.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.004209674999998, 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T04:59:25.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.004186015000002, 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.896 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.004098466000002, 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.901 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T04:59:25.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.004045810000001, 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T04:59:25.905 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T04:59:25.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.909 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.000119918999999, 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.910 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T04:59:25.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.914 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.000094294, 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T04:59:25.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.916 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T04:59:25.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 32.000060900000001, 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T04:59:25.919 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T04:59:25.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.923 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.995270971, 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T04:59:25.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.925 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T04:59:25.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:25.928 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.991876342000001, 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T04:59:25.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.991803665999999, 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.933 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T04:59:25.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T04:59:25.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.988061304999999, 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.938 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:25.942 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.987888313999999, 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T04:59:25.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T04:59:25.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.984045715000001, 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T04:59:25.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.944197737, 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.951 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:25.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T04:59:25.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.939868515000001, 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.956 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T04:59:25.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 31.932139514999999, 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:25.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T04:59:25.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:25.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:25.965 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T04:59:25.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T04:59:25.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.965 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:25.965 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:25.965 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:25.967 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:25.967 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:25.962+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:26.068 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:26.062+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:26.138 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:26.138 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:26.138 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:26.138 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:26.153 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:26.196 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:26.194+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:26.270 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:26.266+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:26.275 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:26.275 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:26.276 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:26.276 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:26.276 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:26.281 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone33 from 0x6f0605a (overlap 0xe0c7a) to 0x9d588 2024-09-13T04:59:26.291 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:26.370 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:26.366+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:26.371 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-13T04:59:26.463 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:26.463 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:26.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:26.463 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.426445185999995, 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:26.468 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.426399548999996, 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:59:26.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:59:26.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:59:26.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:26.473 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.426351464999996, 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:59:26.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.426255382999997, 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:59:26.478 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:26.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:59:26.482 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.426204011999999, 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.483 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:59:26.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:59:26.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.426157958000005, 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:59:26.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:59:26.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:59:26.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.426107896999994, 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.493 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.308692707000006, 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:59:26.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:59:26.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:26.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:59:26.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:26.504 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.217264323999999, 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:59:26.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.217239978999999, 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:26.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.217158626, 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.514 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:59:26.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.518 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.217089095999995, 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:59:26.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.217061819999998, 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.216963375000006, 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.216893807999995, 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:59:26.532 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:59:26.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.536 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.216823333999997, 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:59:26.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:59:26.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.541 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.216797649, 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:26.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:59:26.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.216762461000002, 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:26.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.216737945000006, 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.551 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:26.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 65.216668608000006, 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.555 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.556 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.560 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.560 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:26.562 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:26.562 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:26.558+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:26.662 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:26.658+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:26.698 INFO:teuthology.orchestra.run.smithi001.stdout:5359 trunc from 0x6f0605a to 0xa5b5cac 2024-09-13T04:59:26.717 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:26.718 INFO:teuthology.orchestra.run.smithi001.stdout:5362 punch from 0x57a67d8 to 0x57b570f, (0xef37 bytes) 2024-09-13T04:59:26.718 INFO:teuthology.orchestra.run.smithi001.stdout:5364 punch from 0x4c63767 to 0x4c6f0b1, (0xb94a bytes) 2024-09-13T04:59:26.718 INFO:teuthology.orchestra.run.smithi001.stdout:5365 read 0x3f8a5d2 thru 0x3f90163 (0x5b92 bytes) 2024-09-13T04:59:26.718 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:26.718 INFO:teuthology.orchestra.run.smithi001.stdout:5368 read 0x330a72f thru 0x330f7cd (0x509f bytes) 2024-09-13T04:59:26.718 INFO:teuthology.orchestra.run.smithi001.stdout:5369 writesame 0x16ae2a7 thru 0x16b7159 data_size 0x1b(0x8eb3 bytes) 2024-09-13T04:59:26.726 INFO:teuthology.orchestra.run.smithi001.stdout:5370 write 0x114a972 thru 0x1155db6 (0xb445 bytes) 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:97540 3.1 3:826ca40c:::rbd_data.112a8ecd1681.0000000000000004:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 4194304 write_size 4194304,write 829810~46149 in=46149b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:26.730958+0000", 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0049886230000000002, 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0050463340000000004, 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "waiting for sub ops", 2024-09-13T04:59:26.736 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 97540 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.730958+0000", 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.730958+0000", 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.730955+0000", 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.731161+0000", 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020536099999999999 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.731195+0000", 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4221999999999999e-05 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.731211+0000", 2024-09-13T04:59:26.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6583e-05 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.731306+0000", 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5007999999999996e-05 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.731559+0000", 2024-09-13T04:59:26.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025285500000000002 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.731850+0000", 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029051199999999999 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop_reply(client.4394.0:97540 3.1 e87/24)", 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:26.735423+0000", 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00052367799999999997, 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00070614300000000001, 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.735423+0000", 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.735423+0000", 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.735422+0000", 2024-09-13T04:59:26.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.735433+0000", 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1345999999999999e-05 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.735460+0000", 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7362999999999999e-05 2024-09-13T04:59:26.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:26.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:26.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:26.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:26.735473+0000", 2024-09-13T04:59:26.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.269e-05 2024-09-13T04:59:26.745 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.745 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:26.746 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.746 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:26.746 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-13T04:59:26.746 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 2 2024-09-13T04:59:26.746 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:26.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:26.757 INFO:teuthology.orchestra.run.smithi001.stdout:5371 write 0xd3b92ac thru 0xd3c4421 (0xb176 bytes) 2024-09-13T04:59:26.779 INFO:teuthology.orchestra.run.smithi001.stdout:5372 write 0xe8927f4 thru 0xe8933af (0xbbc bytes) 2024-09-13T04:59:26.786 INFO:teuthology.orchestra.run.smithi001.stdout:5373 read 0x2e78584 thru 0x2e81bec (0x9669 bytes) 2024-09-13T04:59:26.786 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:26.787 INFO:teuthology.orchestra.run.smithi001.stdout:5375 write 0x337a4b0 thru 0x3383ba2 (0x96f3 bytes) 2024-09-13T04:59:26.792 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:26.790+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:26.796 INFO:teuthology.orchestra.run.smithi001.stdout:5376 write 0x9a637ff thru 0x9a6ca43 (0x9245 bytes) 2024-09-13T04:59:26.807 INFO:teuthology.orchestra.run.smithi001.stdout:5377 read 0x5ebab53 thru 0x5ec0c8d (0x613b bytes) 2024-09-13T04:59:26.807 INFO:teuthology.orchestra.run.smithi001.stdout:5378 write 0x44b413f thru 0x44bec37 (0xaaf9 bytes) 2024-09-13T04:59:26.814 INFO:teuthology.orchestra.run.smithi001.stdout:5381 write 0xdad9f4c thru 0xdae1549 (0x75fe bytes) 2024-09-13T04:59:26.847 INFO:teuthology.orchestra.run.smithi001.stdout:5383 trunc from 0xe8933b0 to 0x5204672 2024-09-13T04:59:26.864 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:26.858+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:26.881 INFO:teuthology.orchestra.run.smithi001.stdout:5384 punch from 0x4485b43 to 0x4493753, (0xdc10 bytes) 2024-09-13T04:59:26.881 INFO:teuthology.orchestra.run.smithi001.stdout:5385 read 0x3f20b6c thru 0x3f2a2a3 (0x9738 bytes) 2024-09-13T04:59:26.881 INFO:teuthology.orchestra.run.smithi001.stdout:5386 write 0x2d1e0fd thru 0x2d29e19 (0xbd1d bytes) 2024-09-13T04:59:26.907 INFO:teuthology.orchestra.run.smithi001.stdout:5388 write 0x52eaac1 thru 0x52f5e66 (0xb3a6 bytes) 2024-09-13T04:59:26.911 INFO:teuthology.orchestra.run.smithi001.stdout:5389 writesame 0x169e0a1 thru 0x16a4abb data_size 0x1(0x6a1b bytes) 2024-09-13T04:59:26.924 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:26.924 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:26.924 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:26.925 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:26.925 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:26.938 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:26.965 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:26.958+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:26.973 INFO:teuthology.orchestra.run.smithi001.stdout:5391 trunc from 0x52f5e67 to 0xab29024 2024-09-13T04:59:26.987 INFO:teuthology.orchestra.run.smithi001.stdout:5392 read 0x92cdf7b thru 0x92d4548 (0x65ce bytes) 2024-09-13T04:59:26.988 INFO:teuthology.orchestra.run.smithi001.stdout:5393 write 0x2236e05 thru 0x2239aa9 (0x2ca5 bytes) 2024-09-13T04:59:26.996 INFO:teuthology.orchestra.run.smithi001.stdout:5395 write 0x66dcc2a thru 0x66e40de (0x74b5 bytes) 2024-09-13T04:59:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:5397 read 0x949f467 thru 0x94ac0e4 (0xcc7e bytes) 2024-09-13T04:59:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:5398 read 0xd4b6bb thru 0xd54d95 (0x96db bytes) 2024-09-13T04:59:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:5399 punch from 0x49321cd to 0x493d488, (0xb2bb bytes) 2024-09-13T04:59:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:5400 punch from 0x9ff48b4 to 0x9fffc47, (0xb393 bytes) 2024-09-13T04:59:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:5401 read 0x6ab84fb thru 0x6ab8f06 (0xa0c bytes) 2024-09-13T04:59:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:5402 clone 35 order 20 su 131072 sc 2 2024-09-13T04:59:27.006 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:27.006 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:27.006 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T04:59:27.006 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.379358 2024-09-13T04:59:27.006 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.3793583539598735 2024-09-13T04:59:27.059 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:27.059 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.258064666999999, 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.060 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:27.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.258030413, 2024-09-13T04:59:27.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:27.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257999511999998, 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:27.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.073 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257947631, 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:27.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:27.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:27.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.078 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257915828000002, 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:27.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:27.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:27.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257881875000002, 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.083 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.087 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257834139000003, 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:27.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257794576999999, 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.092 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.096 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257746883000003, 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:27.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:27.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:27.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257683800000002, 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.101 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257647366, 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257613442999997, 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257545229999998, 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257511524000002, 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257461331999998, 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257155298999997, 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257106096999998, 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257057983000003, 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.257006506000003, 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 63.256971425000003, 2024-09-13T04:59:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:27.149 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.150 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.150 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:27.151 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:27.152 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:27.146+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:27.182 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:59:27.252 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:27.250+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:27.320 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:27.320 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:27.320 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:27.335 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:27.377 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:27.374+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:27.454 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:27.450+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:27.497 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:27.497 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:27.497 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:27.497 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:27.498 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:27.511 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:27.555 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:27.550+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49885 3.b 3:d43de71f:::rbd_data.112a40bf6452.0000000000000003:f [sparse-read 0~1843200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.605679903000002, 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015118388, 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.637 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49885 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028150+0000", 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028157+0000", 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5510000000000002e-06 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028174+0000", 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6631000000000001e-05 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028183+0000", 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6100000000000006e-06 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028308+0000", 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012532600000000001 2024-09-13T04:59:27.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028557+0000", 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024922200000000001 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.043270+0000", 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014712578 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49972 3.b 3:d2a26831:::rbd_data.112ae3a54590.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.454414497000002, 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021756569999999999, 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49972 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179414+0000", 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:27.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180536+0000", 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001122381 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180563+0000", 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6211999999999999e-05 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180582+0000", 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9630999999999999e-05 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180662+0000", 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9411000000000003e-05 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180947+0000", 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000285166 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181294+0000", 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034697800000000001 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200928+0000", 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019634766000000001 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201089+0000", 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016092299999999999 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201142+0000", 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2855000000000001e-05 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201173+0000", 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1096999999999997e-05 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49973 3.b 3:d63ae98e:::rbd_data.112ae3a54590.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:27.647 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.453220958000003, 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026409885000000001, 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49973 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:27.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180608+0000", 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181851+0000", 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012428630000000001 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181894+0000", 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2710000000000003e-05 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181909+0000", 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5092999999999999e-05 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.186892+0000", 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049824229999999997 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187123+0000", 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023144999999999999 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187416+0000", 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029303400000000001 2024-09-13T04:59:27.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200934+0000", 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013517714 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206897+0000", 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059634880000000003 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206972+0000", 2024-09-13T04:59:27.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5165000000000003e-05 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207020+0000", 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7821000000000002e-05 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49975 3.b 3:d71a7fd7:::rbd_data.112ae3a54590.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1826816~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 51.451562197999998, 2024-09-13T04:59:27.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025198755, 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49975 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182267+0000", 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183414+0000", 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011469270000000001 2024-09-13T04:59:27.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183442+0000", 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8189000000000001e-05 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183458+0000", 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6073999999999999e-05 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.195878+0000", 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01241922 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196039+0000", 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161058 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196279+0000", 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240676 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201295+0000", 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0050153999999999997 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207382+0000", 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0060871650000000003 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207432+0000", 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9882000000000001e-05 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207468+0000", 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6140000000000003e-05 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79481 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.659 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.182496290000003, 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010622207, 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79481 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:27.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451331+0000", 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451350+0000", 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8448999999999999e-05 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451425+0000", 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4929e-05 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451448+0000", 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3618999999999999e-05 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451655+0000", 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020621600000000001 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451671+0000", 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6013000000000001e-05 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461479+0000", 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098082340000000007 2024-09-13T04:59:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461702+0000", 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022309900000000001 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461957+0000", 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000255171 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79525 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.182342083000002, 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010933693, 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79525 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451488+0000", 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451497+0000", 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8359999999999998e-06 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451527+0000", 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0015999999999999e-05 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451543+0000", 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6059999999999999e-05 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451797+0000", 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025463100000000001 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451810+0000", 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2999999999999999e-05 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462017+0000", 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010207101999999999 2024-09-13T04:59:27.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462210+0000", 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192649 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462423+0000", 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021292899999999999 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.669 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79695 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.178637997000003, 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010472398000000001, 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79695 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455192+0000", 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455198+0000", 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9399999999999999e-06 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455215+0000", 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.632e-05 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455229+0000", 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4146e-05 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455854+0000", 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00062516999999999998 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455860+0000", 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7300000000000002e-06 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.464832+0000", 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0089722559999999996 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465369+0000", 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00053748100000000003 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465666+0000", 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029624799999999999 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.674 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79696 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.178590604, 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011064011, 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79696 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:27.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455240+0000", 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455246+0000", 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5830000000000003e-06 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455264+0000", 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7450000000000001e-05 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455277+0000", 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3016e-05 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455886+0000", 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000609637 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455892+0000", 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5199999999999997e-06 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465781+0000", 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098892089999999995 2024-09-13T04:59:27.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466021+0000", 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024036500000000001 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466305+0000", 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028331100000000001 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79718 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.178542741999998, 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011634479, 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79718 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455288+0000", 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455293+0000", 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455310+0000", 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6393e-05 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455323+0000", 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3013000000000001e-05 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455918+0000", 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059522399999999999 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455923+0000", 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4060000000000004e-06 2024-09-13T04:59:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466385+0000", 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010462154 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466593+0000", 2024-09-13T04:59:27.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020723900000000001 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466923+0000", 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00033036300000000002 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79737 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.685 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.178497415999999, 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011808300000000001, 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79737 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.686 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455333+0000", 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455339+0000", 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7570000000000001e-06 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455355+0000", 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.632e-05 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455368+0000", 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2588999999999999e-05 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455948+0000", 2024-09-13T04:59:27.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058065099999999995 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455954+0000", 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466957+0000", 2024-09-13T04:59:27.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011003361 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467033+0000", 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5644999999999998e-05 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467142+0000", 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109527 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79757 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 393216~38443] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.178452104000002, 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013373497999999999, 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.692 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79757 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455378+0000", 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455384+0000", 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6200000000000004e-06 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455400+0000", 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5973000000000001e-05 2024-09-13T04:59:27.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455412+0000", 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2689e-05 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455979+0000", 2024-09-13T04:59:27.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00056651800000000001 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455984+0000", 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.49e-06 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467237+0000", 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011252376999999999 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467453+0000", 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000216338 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468753+0000", 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012995459999999999 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79776 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:27.698 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.178407258, 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013932595000000001, 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79776 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:27.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455423+0000", 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455429+0000", 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455446+0000", 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6877000000000001e-05 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455459+0000", 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2866e-05 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456009+0000", 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000550838 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456015+0000", 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6060000000000002e-06 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468961+0000", 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012946301 2024-09-13T04:59:27.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469181+0000", 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000220075 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469357+0000", 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000175279 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:79796 3.0 e60/24 3:003e46ae:::rbd_data.112acc4fa3fd.0000000000000000:head v 60'366, mlcod=58'364)", 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.143074581, 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025297777, 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490755+0000", 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494145+0000", 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033899300000000002 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494187+0000", 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1950999999999999e-05 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494201+0000", 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4134000000000001e-05 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494267+0000", 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5815999999999995e-05 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494296+0000", 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8597000000000001e-05 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516017+0000", 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021721285 2024-09-13T04:59:27.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516054+0000", 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7389999999999999e-05 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79853 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.090094395000001, 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043236985999999998, 2024-09-13T04:59:27.708 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79853 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543734+0000", 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548313+0000", 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045782449999999999 2024-09-13T04:59:27.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548339+0000", 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6948999999999999e-05 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548356+0000", 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6160000000000001e-05 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548459+0000", 2024-09-13T04:59:27.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000103453 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548718+0000", 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025891599999999999 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.549442+0000", 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00072383699999999996 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586544+0000", 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037102028000000002 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586859+0000", 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000315237 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586924+0000", 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5315999999999996e-05 2024-09-13T04:59:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586974+0000", 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9481000000000003e-05 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81749 3.b 3:d52afe2d:::rbd_data.112a86dc1e25.000000000000000d:head [stat,delete] snapc 18=[18] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.561627305999998, 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010905319, 2024-09-13T04:59:27.714 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81749 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072201+0000", 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072215+0000", 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3859000000000001e-05 2024-09-13T04:59:27.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072244+0000", 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8612999999999999e-05 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072265+0000", 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0893000000000001e-05 2024-09-13T04:59:27.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.073443+0000", 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011780359999999999 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.073753+0000", 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031037700000000003 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.074309+0000", 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00055601200000000004 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.076960+0000", 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002650856 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.082880+0000", 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005919901 2024-09-13T04:59:27.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.083013+0000", 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000133168 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.083109+0000", 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6163999999999996e-05 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81735 3.8 e64/24 3:148b359f:::rbd_data.112a86dc1e25.0000000000000006:head v 64'139, mlcod=64'137)", 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.556111086000001, 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011631831, 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077718+0000", 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078104+0000", 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000385852 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078144+0000", 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9858999999999999e-05 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078163+0000", 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8666999999999999e-05 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078282+0000", 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000119784 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078334+0000", 2024-09-13T04:59:27.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1712000000000002e-05 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.089290+0000", 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010955515000000001 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.089352+0000", 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2389000000000002e-05 2024-09-13T04:59:27.725 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81782 3.c e64/24 3:33317f5e:::rbd_data.112a86dc1e25.000000000000000b:head v 64'410, mlcod=63'408)", 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.538779189, 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012792019999999999, 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.726 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095051+0000", 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095635+0000", 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058456600000000001 2024-09-13T04:59:27.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095660+0000", 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4374000000000001e-05 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095670+0000", 2024-09-13T04:59:27.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0631999999999999e-05 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095743+0000", 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3015000000000005e-05 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095775+0000", 2024-09-13T04:59:27.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2092999999999999e-05 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107755+0000", 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011979949 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107844+0000", 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8858999999999997e-05 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81874 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [sparse-read 0~2423869] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:27.732 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 29.366113625000001, 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015773423000000002, 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81874 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267715+0000", 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267729+0000", 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4399999999999999e-05 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267761+0000", 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2112e-05 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.735 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267776+0000", 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4823e-05 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267903+0000", 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000127014 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.268135+0000", 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023146899999999999 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.283491+0000", 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015356491 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:96142 3.0 e84/24 3:05f91f13:::rbd_data.112a487824ce.0000000000000000:head v 84'514, mlcod=82'512)", 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.3857721700000001, 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013444012999999999, 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.737 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:27.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248058+0000", 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248437+0000", 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000378945 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248455+0000", 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7791999999999998e-05 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248467+0000", 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1568e-05 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248575+0000", 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108637 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248605+0000", 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9471999999999998e-05 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.261461+0000", 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012856627000000001 2024-09-13T04:59:27.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.261503+0000", 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656000000000002e-05 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:96241 3.b 3:d219a614:::rbd_data.112a3f73279b.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3069572~54962] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 5.1090214659999997, 2024-09-13T04:59:27.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014882869999999999, 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 96241 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524806+0000", 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525284+0000", 2024-09-13T04:59:27.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047714799999999999 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525320+0000", 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6219e-05 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525337+0000", 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7183000000000001e-05 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525421+0000", 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3944999999999997e-05 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525613+0000", 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019242099999999999 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525898+0000", 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028419399999999998 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.533236+0000", 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073389190000000002 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539592+0000", 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.006355688 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539646+0000", 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4360000000000001e-05 2024-09-13T04:59:27.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539693+0000", 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6205999999999998e-05 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:27.748 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:27.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:27.750 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:27.746+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:27.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:27.851 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:27.846+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:27.884 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:27.884 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:27.899 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:27.950 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:27.950 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:27.952 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:27.946+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:27.967 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:27.967 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:27.983 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:27.984 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:28.000 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:28.000 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:28.020 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:28.021 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:28.040 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:28.052 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:28.050+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:28.056 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:28.057 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:28.060 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:28.060 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:28.072 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:28.081 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:28.081 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:28.153 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:28.150+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:28.162 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.6 primary-affinity to 0.379358 (8248612) 2024-09-13T04:59:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:59:28.219 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:28.219 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:28.234 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:28.254 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:28.250+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:28.355 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:28.350+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:28.394 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:28.395 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:28.409 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:28.456 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:28.454+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:28.529 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:28.529 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:28.544 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:28.557 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:28.554+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:28.658 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:28.654+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:28.670 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:28.670 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:28.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:28.758 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:28.754+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:28.810 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:28.810 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:28.825 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:28.873 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:28.870+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:28.945 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:28.945 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:28.961 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:28.961 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:28.954+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:29.061 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:29.054+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:29.138 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:29.182 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:29.178+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:29.250 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone34 from 0xab29024 (overlap 0x5204672) to 0x20ce8fa 2024-09-13T04:59:29.263 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:29.258+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:29.321 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:29.322 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:29.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:29.385 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:29.382+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:29.421 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-13T04:59:29.432 INFO:teuthology.orchestra.run.smithi001.stdout:5404 write 0xfe4bbe thru 0xff1671 (0xcab4 bytes) 2024-09-13T04:59:29.452 INFO:teuthology.orchestra.run.smithi001.stdout:5405 writesame 0x33c0029 thru 0x33ced2f data_size 0x1(0xed07 bytes) 2024-09-13T04:59:29.465 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:29.458+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:29.468 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:29.468 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:29.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:29.492 INFO:teuthology.orchestra.run.smithi001.stdout:5407 writesame 0x1696a1 thru 0x175f6f data_size 0x1(0xc8cf bytes) 2024-09-13T04:59:29.566 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:29.562+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:29.567 INFO:teuthology.orchestra.run.smithi001.stdout:5408 read 0x9727bf8 thru 0x972b18f (0x3598 bytes) 2024-09-13T04:59:29.587 INFO:teuthology.orchestra.run.smithi001.stdout:5409 writesame 0xcd99870 thru 0xcda04d3 data_size 0xe(0x6c64 bytes) 2024-09-13T04:59:29.587 INFO:teuthology.orchestra.run.smithi001.stdout:5410 write 0x44c3c0f thru 0x44d0fa2 (0xd394 bytes) 2024-09-13T04:59:29.605 INFO:teuthology.orchestra.run.smithi001.stdout:5411 writesame 0x712b03b thru 0x713607b data_size 0x1(0xb041 bytes) 2024-09-13T04:59:29.625 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:29.625 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:29.641 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:29.674 INFO:teuthology.orchestra.run.smithi001.stdout:5412 punch from 0xa1f2a6f to 0xa201b93, (0xf124 bytes) 2024-09-13T04:59:29.674 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:29.674 INFO:teuthology.orchestra.run.smithi001.stdout:5416 read 0x8d6c4f2 thru 0x8d6fe4e (0x395d bytes) 2024-09-13T04:59:29.674 INFO:teuthology.orchestra.run.smithi001.stdout:5417 write 0x1afe1d2 thru 0x1b0ad0d (0xcb3c bytes) 2024-09-13T04:59:29.685 INFO:teuthology.orchestra.run.smithi001.stdout:5419 punch from 0x86e2100 to 0x86ed124, (0xb024 bytes) 2024-09-13T04:59:29.688 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:29.686+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:29.693 INFO:teuthology.orchestra.run.smithi001.stdout:5420 write 0xcdf2965 thru 0xcdf4bee (0x228a bytes) 2024-09-13T04:59:29.697 INFO:teuthology.orchestra.run.smithi001.stdout:5422 writesame 0x6c57cfb thru 0x6c5faed data_size 0x1(0x7df3 bytes) 2024-09-13T04:59:29.745 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:29.745 INFO:teuthology.orchestra.run.smithi001.stdout:5426 write 0x5446d0d thru 0x5455f42 (0xf236 bytes) 2024-09-13T04:59:29.768 INFO:teuthology.orchestra.run.smithi001.stdout:5428 write 0x5263566 thru 0x526533f (0x1dda bytes) 2024-09-13T04:59:29.769 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:29.766+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:29.777 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:29.778 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:29.780 INFO:teuthology.orchestra.run.smithi001.stdout:5429 write 0x4eb5180 thru 0x4eb7e9c (0x2d1d bytes) 2024-09-13T04:59:29.788 INFO:teuthology.orchestra.run.smithi001.stdout:5430 read 0x50c5d58 thru 0x50ccf80 (0x7229 bytes) 2024-09-13T04:59:29.789 INFO:teuthology.orchestra.run.smithi001.stdout:5431 write 0x280b07b thru 0x281007f (0x5005 bytes) 2024-09-13T04:59:29.793 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:29.798 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:29.798 INFO:teuthology.orchestra.run.smithi001.stdout:5433 punch from 0x571f912 to 0x572a9af, (0xb09d bytes) 2024-09-13T04:59:29.798 INFO:teuthology.orchestra.run.smithi001.stdout:5434 writesame 0x3514dfc thru 0x3518a8d data_size 0x1(0x3c92 bytes) 2024-09-13T04:59:29.825 INFO:teuthology.orchestra.run.smithi001.stdout:5435 writesame 0x179ba0a thru 0x179ef2d data_size 0x26(0x3524 bytes) 2024-09-13T04:59:29.833 INFO:teuthology.orchestra.run.smithi001.stdout:5437 write 0x22c5f8e thru 0x22ceb6e (0x8be1 bytes) 2024-09-13T04:59:29.848 INFO:teuthology.orchestra.run.smithi001.stdout:5438 trunc from 0xcdf4bef to 0xbe5961e 2024-09-13T04:59:29.880 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:29.878+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:29.887 INFO:teuthology.orchestra.run.smithi001.stdout:5440 punch from 0x1c1f0ca to 0x1c28261, (0x9197 bytes) 2024-09-13T04:59:29.887 INFO:teuthology.orchestra.run.smithi001.stdout:5442 write 0x5e7b456 thru 0x5e855c5 (0xa170 bytes) 2024-09-13T04:59:29.896 INFO:teuthology.orchestra.run.smithi001.stdout:5443 punch from 0xae9fd3e to 0xaea741b, (0x76dd bytes) 2024-09-13T04:59:29.911 INFO:teuthology.orchestra.run.smithi001.stdout:5444 write 0xcddaba3 thru 0xcddc8ef (0x1d4d bytes) 2024-09-13T04:59:29.918 INFO:teuthology.orchestra.run.smithi001.stdout:5445 read 0x6afe067 thru 0x6b0c547 (0xe4e1 bytes) 2024-09-13T04:59:29.940 INFO:teuthology.orchestra.run.smithi001.stdout:5446 trunc from 0xcddc8f0 to 0x5327221 2024-09-13T04:59:29.970 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:29.970 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:29.970 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:29.966+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:29.984 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:30.013 INFO:teuthology.orchestra.run.smithi001.stdout:5448 writesame 0xeb56383 thru 0xeb5785c data_size 0x11(0x14da bytes) 2024-09-13T04:59:30.029 INFO:teuthology.orchestra.run.smithi001.stdout:5449 trunc from 0xeb5785d to 0x7e415d6 2024-09-13T04:59:30.066 INFO:teuthology.orchestra.run.smithi001.stdout:5450 write 0xcd2867a thru 0xcd367fa (0xe181 bytes) 2024-09-13T04:59:30.071 INFO:teuthology.orchestra.run.smithi001.stdout:5452 read 0x1bdb6c2 thru 0x1be98c3 (0xe202 bytes) 2024-09-13T04:59:30.071 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:30.066+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:30.072 INFO:teuthology.orchestra.run.smithi001.stdout:5454 write 0x13df3e4 thru 0x13e656f (0x718c bytes) 2024-09-13T04:59:30.083 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:30.083 INFO:teuthology.orchestra.run.smithi001.stdout:5458 trunc from 0xcd367fb to 0xce56c53 2024-09-13T04:59:30.099 INFO:teuthology.orchestra.run.smithi001.stdout:5459 trunc from 0xce56c53 to 0xb5dc875 2024-09-13T04:59:30.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:30.131 INFO:teuthology.orchestra.run.smithi001.stdout:5461 write 0xd8560a1 thru 0xd85fb24 (0x9a84 bytes) 2024-09-13T04:59:30.138 INFO:teuthology.orchestra.run.smithi001.stdout:5462 writesame 0x1e3b6f0 thru 0x1e45ee0 data_size 0x33(0xa7f1 bytes) 2024-09-13T04:59:30.146 INFO:teuthology.orchestra.run.smithi001.stdout:5463 write 0xd596eac thru 0xd5a1842 (0xa997 bytes) 2024-09-13T04:59:30.158 INFO:teuthology.orchestra.run.smithi001.stdout:5464 read 0xcbb40a8 thru 0xcbc3f73 (0xfecc bytes) 2024-09-13T04:59:30.158 INFO:teuthology.orchestra.run.smithi001.stdout:5465 punch from 0x65615d6 to 0x6570b29, (0xf553 bytes) 2024-09-13T04:59:30.158 INFO:teuthology.orchestra.run.smithi001.stdout:5466 punch from 0x3c7584a to 0x3c7edbe, (0x9574 bytes) 2024-09-13T04:59:30.158 INFO:teuthology.orchestra.run.smithi001.stdout:5468 punch from 0x992cd04 to 0x993116a, (0x4466 bytes) 2024-09-13T04:59:30.158 INFO:teuthology.orchestra.run.smithi001.stdout:5469 write 0x57f5235 thru 0x57f6687 (0x1453 bytes) 2024-09-13T04:59:30.160 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:30.161 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:30.163 INFO:teuthology.orchestra.run.smithi001.stdout:5472 punch from 0x47a1c6f to 0x47ae726, (0xcab7 bytes) 2024-09-13T04:59:30.164 INFO:teuthology.orchestra.run.smithi001.stdout:5473 read 0xe8e5e6 thru 0xe9ce97 (0xe8b2 bytes) 2024-09-13T04:59:30.164 INFO:teuthology.orchestra.run.smithi001.stdout:5474 write 0xc8efc6a thru 0xc8f2869 (0x2c00 bytes) 2024-09-13T04:59:30.169 INFO:teuthology.orchestra.run.smithi001.stdout:5475 writesame 0x2f79a94 thru 0x2f8138d data_size 0x5f(0x78fa bytes) 2024-09-13T04:59:30.173 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:30.166+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:30.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:30.179 INFO:teuthology.orchestra.run.smithi001.stdout:5477 read 0xa1dcbd5 thru 0xa1e653a (0x9966 bytes) 2024-09-13T04:59:30.190 INFO:teuthology.orchestra.run.smithi001.stdout:5478 writesame 0xeb49b80 thru 0xeb4d42d data_size 0x5(0x38ae bytes) 2024-09-13T04:59:30.191 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:59:30.195 INFO:teuthology.orchestra.run.smithi001.stdout:5479 trunc from 0xeb4d42e to 0xbb96682 2024-09-13T04:59:30.211 INFO:teuthology.orchestra.run.smithi001.stdout:5480 writesame 0x612bf86 thru 0x612fb30 data_size 0xd(0x3bab bytes) 2024-09-13T04:59:30.215 INFO:teuthology.orchestra.run.smithi001.stdout:5482 read 0x56c456c thru 0x56cdda6 (0x983b bytes) 2024-09-13T04:59:30.216 INFO:teuthology.orchestra.run.smithi001.stdout:5487 trunc from 0xbb96682 to 0xb6ae1ad 2024-09-13T04:59:30.225 INFO:teuthology.orchestra.run.smithi001.stdout:5488 trunc from 0xb6ae1ad to 0xa9b5b85 2024-09-13T04:59:30.244 INFO:teuthology.orchestra.run.smithi001.stdout:5490 trunc from 0xa9b5b85 to 0x4a55c7f 2024-09-13T04:59:30.262 INFO:teuthology.orchestra.run.smithi001.stdout:5491 read 0x1bb8896 thru 0x1bb94f5 (0xc60 bytes) 2024-09-13T04:59:30.273 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:30.273 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:30.273 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:30.270+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:30.276 INFO:teuthology.orchestra.run.smithi001.stdout:5492 write 0x950ceb7 thru 0x9510763 (0x38ad bytes) 2024-09-13T04:59:30.280 INFO:teuthology.orchestra.run.smithi001.stdout:5493 read 0x4578ac6 thru 0x457e8dc (0x5e17 bytes) 2024-09-13T04:59:30.281 INFO:teuthology.orchestra.run.smithi001.stdout:5494 writesame 0x4e9b47c thru 0x4e9e809 data_size 0x1(0x338e bytes) 2024-09-13T04:59:30.293 INFO:teuthology.orchestra.run.smithi001.stdout:5495 read 0x6350040 thru 0x635a289 (0xa24a bytes) 2024-09-13T04:59:30.294 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:30.294 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:30.303 INFO:teuthology.orchestra.run.smithi001.stdout:5496 write 0xb2a1189 thru 0xb2a934f (0x81c7 bytes) 2024-09-13T04:59:30.306 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:30.306 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:30.312 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:30.312 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:30.319 INFO:teuthology.orchestra.run.smithi001.stdout:5497 trunc from 0xb2a9350 to 0x442efe2 2024-09-13T04:59:30.320 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:30.331 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:30.331 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:30.340 INFO:teuthology.orchestra.run.smithi001.stdout:5498 punch from 0x36ea3de to 0x36f3ac6, (0x96e8 bytes) 2024-09-13T04:59:30.340 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:30.351 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:30.351 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:30.361 INFO:teuthology.orchestra.run.smithi001.stdout:5501 write 0xd86f428 thru 0xd873163 (0x3d3c bytes) 2024-09-13T04:59:30.365 INFO:teuthology.orchestra.run.smithi001.stdout:5502 write 0x70f1748 thru 0x70f39ce (0x2287 bytes) 2024-09-13T04:59:30.370 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:30.370 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:30.374 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:30.370+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:30.388 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:30.389 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:30.408 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:30.440 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:30.440 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:30.440 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:30.440 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:30.440 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:30.455 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:30.501 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:30.498+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:30.577 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:30.574+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:30.578 INFO:teuthology.orchestra.run.smithi001.stdout:5503 writesame 0x4ed9238 thru 0x4edb2ec data_size 0x1(0x20b5 bytes) 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.544710409999993, 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.582 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.583 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.584 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.585 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.586 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.544664772999994, 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.587 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.588 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.590 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T04:59:30.591 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.544616688999994, 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.592 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.593 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T04:59:30.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.595 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:30.596 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.544520606999995, 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.597 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.598 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.599 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.600 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.544469235999998, 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.601 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.602 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T04:59:30.603 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T04:59:30.604 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.605 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.544423182000003, 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.606 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.607 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.608 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.609 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.544373121000007, 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.610 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T04:59:30.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T04:59:30.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T04:59:30.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.615 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 74.426957931000004, 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.616 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T04:59:30.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T04:59:30.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31466 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1720320~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335529547999997, 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081461765000000005, 2024-09-13T04:59:30.622 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31466 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243125+0000", 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243124+0000", 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243127+0000", 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5729999999999999e-06 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243135+0000", 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3319999999999992e-06 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243143+0000", 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2300000000000002e-06 2024-09-13T04:59:30.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324350+0000", 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081206873999999998 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324458+0000", 2024-09-13T04:59:30.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010806000000000001 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324586+0000", 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012863600000000001 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335505202999997, 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.628 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T04:59:30.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:30.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T04:59:30.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T04:59:30.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T04:59:30.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T04:59:30.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T04:59:30.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T04:59:30.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T04:59:30.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.634 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.635 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.635 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:30.635 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335423849999998, 2024-09-13T04:59:30.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T04:59:30.635 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.635 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:30.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T04:59:30.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T04:59:30.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T04:59:30.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T04:59:30.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T04:59:30.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T04:59:30.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T04:59:30.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T04:59:30.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335354319999993, 2024-09-13T04:59:30.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T04:59:30.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.646 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335327043999996, 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T04:59:30.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T04:59:30.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.651 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335228599000004, 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T04:59:30.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T04:59:30.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.657 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335159032000007, 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T04:59:30.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T04:59:30.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T04:59:30.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T04:59:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335088557999995, 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T04:59:30.663 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.664 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.664 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.664 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.664 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.664 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T04:59:30.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:30.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T04:59:30.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T04:59:30.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T04:59:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T04:59:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335062872999998, 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T04:59:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T04:59:30.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:30.675 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335027685, 2024-09-13T04:59:30.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T04:59:30.676 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.676 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.676 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T04:59:30.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T04:59:30.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.335003169000004, 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T04:59:30.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T04:59:30.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T04:59:30.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.685 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T04:59:30.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T04:59:30.685 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 69.334933832000004, 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:30.686 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:30.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T04:59:30.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:59:30.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:30.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T04:59:30.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T04:59:30.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:30.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T04:59:30.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T04:59:30.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:30.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T04:59:30.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T04:59:30.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:30.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:30.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:30.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T04:59:30.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T04:59:30.691 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.691 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.692 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:30.692 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:30.692 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:30.698 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:30.698 INFO:teuthology.orchestra.run.smithi001.stdout:5504 read 0x925346a thru 0x925ff3d (0xcad4 bytes) 2024-09-13T04:59:30.698 INFO:teuthology.orchestra.run.smithi001.stdout:5505 write 0xb8c0862 thru 0xb8c0b7c (0x31b bytes) 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5507 write 0xcb479a0 thru 0xcb4d088 (0x56e9 bytes) 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5508 punch from 0xb7ff10e to 0xb803890, (0x4782 bytes) 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5509 punch from 0x5ef891c to 0x5f02ee8, (0xa5cc bytes) 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5512 punch from 0x7d8a299 to 0x7d8e33a, (0x40a1 bytes) 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5514 writesame 0xd89e408 thru 0xd8a478d data_size 0x1(0x6386 bytes) 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5515 write 0x725c2 thru 0x72d1c (0x75b bytes) 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5519 trunc from 0xd8a478e to 0x37ef185 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5520 write 0x924735 thru 0x924aa0 (0x36c bytes) 2024-09-13T04:59:30.699 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:30.690+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:30.699 INFO:teuthology.orchestra.run.smithi001.stdout:5521 clone 36 order 22 su 131072 sc 4 2024-09-13T04:59:30.794 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:30.790+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:30.868 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:30.868 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:30.868 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:30.868 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:30.882 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:30.895 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:30.890+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:30.996 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:30.990+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:31.004 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:31.004 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:31.004 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:31.004 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:31.004 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:31.018 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:31.097 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:31.090+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:59:31.193 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:31.193 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:31.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:31.193 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.391245979000004, 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:31.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.391211725000005, 2024-09-13T04:59:31.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:31.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.391180824000003, 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.203 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.391128942999998, 2024-09-13T04:59:31.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:31.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:31.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.391097139999999, 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.213 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:31.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.391063187, 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.217 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:31.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.220 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.391015451000001, 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:31.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:31.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.225 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390975889000003, 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.226 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:31.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390928195000001, 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.231 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:31.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.234 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:31.235 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390865112, 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:31.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:31.239 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390828678000005, 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.240 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:31.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390794755000002, 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:31.244 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:31.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:31.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390726541999996, 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:31.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390692835999999, 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.253 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:31.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.257 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390642643999996, 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.258 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:31.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390336610999995, 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.262 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.266 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390287408999995, 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:31.270 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390239295000001, 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.271 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:31.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390187818000001, 2024-09-13T04:59:31.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:31.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.279 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 67.390152736999994, 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.280 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:31.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:31.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.284 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:31.286 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:31.287 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:31.282+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:31.315 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone35 from 0x37ef185 (overlap 0x37ef185) to 0x597e8d 2024-09-13T04:59:31.386 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:31.382+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:31.471 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:31.471 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:31.472 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:31.485 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:31.487 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:31.482+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:31.527 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-13T04:59:31.588 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:31.586+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:31.607 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:31.607 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:31.607 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:31.608 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:31.608 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:31.621 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:31.627 INFO:teuthology.orchestra.run.smithi001.stdout:5523 punch from 0x2a9aaa1 to 0x2aa3955, (0x8eb4 bytes) 2024-09-13T04:59:31.651 INFO:teuthology.orchestra.run.smithi001.stdout:5524 write 0x54d8983 thru 0x54dafb0 (0x262e bytes) 2024-09-13T04:59:31.657 INFO:teuthology.orchestra.run.smithi001.stdout:5525 punch from 0x3e04815 to 0x3e079fe, (0x31e9 bytes) 2024-09-13T04:59:31.658 INFO:teuthology.orchestra.run.smithi001.stdout:5527 clone 37 order 19 su 65536 sc 3 2024-09-13T04:59:31.709 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:31.706+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:31.784 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:31.784 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:31.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:31.784 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:31.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49885 3.b 3:d43de71f:::rbd_data.112a40bf6452.0000000000000003:f [sparse-read 0~1843200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 55.752714322000003, 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015118388, 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49885 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028150+0000", 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028157+0000", 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5510000000000002e-06 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028174+0000", 2024-09-13T04:59:31.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6631000000000001e-05 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028183+0000", 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6100000000000006e-06 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028308+0000", 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012532600000000001 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028557+0000", 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024922200000000001 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.043270+0000", 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014712578 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.789 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49972 3.b 3:d2a26831:::rbd_data.112ae3a54590.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 55.601448916000002, 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021756569999999999, 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49972 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.790 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179414+0000", 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180536+0000", 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001122381 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180563+0000", 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6211999999999999e-05 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180582+0000", 2024-09-13T04:59:31.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9630999999999999e-05 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180662+0000", 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9411000000000003e-05 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180947+0000", 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000285166 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181294+0000", 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034697800000000001 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200928+0000", 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019634766000000001 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201089+0000", 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016092299999999999 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201142+0000", 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2855000000000001e-05 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201173+0000", 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1096999999999997e-05 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49973 3.b 3:d63ae98e:::rbd_data.112ae3a54590.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 55.600255377000003, 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026409885000000001, 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49973 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:31.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180608+0000", 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181851+0000", 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012428630000000001 2024-09-13T04:59:31.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181894+0000", 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2710000000000003e-05 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181909+0000", 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5092999999999999e-05 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.186892+0000", 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049824229999999997 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187123+0000", 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023144999999999999 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187416+0000", 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029303400000000001 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200934+0000", 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013517714 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206897+0000", 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059634880000000003 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206972+0000", 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5165000000000003e-05 2024-09-13T04:59:31.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207020+0000", 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7821000000000002e-05 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49975 3.b 3:d71a7fd7:::rbd_data.112ae3a54590.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1826816~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 55.598596616999998, 2024-09-13T04:59:31.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025198755, 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49975 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182267+0000", 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183414+0000", 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011469270000000001 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183442+0000", 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8189000000000001e-05 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183458+0000", 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6073999999999999e-05 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.195878+0000", 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01241922 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196039+0000", 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161058 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196279+0000", 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240676 2024-09-13T04:59:31.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201295+0000", 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0050153999999999997 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207382+0000", 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0060871650000000003 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207432+0000", 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9882000000000001e-05 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207468+0000", 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6140000000000003e-05 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79481 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:31.808 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.329530708999997, 2024-09-13T04:59:31.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010622207, 2024-09-13T04:59:31.809 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.809 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.809 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.809 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79481 2024-09-13T04:59:31.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451335+0000", 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451331+0000", 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451350+0000", 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8448999999999999e-05 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451425+0000", 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4929e-05 2024-09-13T04:59:31.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451448+0000", 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3618999999999999e-05 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451655+0000", 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020621600000000001 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op must wait for map", 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451671+0000", 2024-09-13T04:59:31.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6013000000000001e-05 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461479+0000", 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098082340000000007 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461702+0000", 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022309900000000001 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.461957+0000", 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000255171 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79525 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.329376502000002, 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010933693, 2024-09-13T04:59:31.817 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79525 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451489+0000", 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451488+0000", 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451497+0000", 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8359999999999998e-06 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451527+0000", 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0015999999999999e-05 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451543+0000", 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6059999999999999e-05 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451797+0000", 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025463100000000001 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.451810+0000", 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2999999999999999e-05 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462017+0000", 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010207101999999999 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462210+0000", 2024-09-13T04:59:31.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000192649 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.462423+0000", 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021292899999999999 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79695 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.325672416000003, 2024-09-13T04:59:31.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010472398000000001, 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79695 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:31.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455193+0000", 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455192+0000", 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455198+0000", 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9399999999999999e-06 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455215+0000", 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.632e-05 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455229+0000", 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4146e-05 2024-09-13T04:59:31.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455854+0000", 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00062516999999999998 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455860+0000", 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7300000000000002e-06 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.464832+0000", 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0089722559999999996 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465369+0000", 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00053748100000000003 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465666+0000", 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029624799999999999 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.827 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79696 3.b 3:d0f1ad2a:::rbd_data.112aaf1f27a8.0000000000000011:16 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.325625023000001, 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011064011, 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.828 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79696 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455241+0000", 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455240+0000", 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455246+0000", 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5830000000000003e-06 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455264+0000", 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7450000000000001e-05 2024-09-13T04:59:31.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455277+0000", 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3016e-05 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455886+0000", 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000609637 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455892+0000", 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5199999999999997e-06 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.465781+0000", 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098892089999999995 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466021+0000", 2024-09-13T04:59:31.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024036500000000001 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466305+0000", 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028331100000000001 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79718 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:31.833 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.325577160999998, 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011634479, 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79718 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:31.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455289+0000", 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455288+0000", 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455293+0000", 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455310+0000", 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6393e-05 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455323+0000", 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3013000000000001e-05 2024-09-13T04:59:31.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455918+0000", 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00059522399999999999 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455923+0000", 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4060000000000004e-06 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466385+0000", 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010462154 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466593+0000", 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00020723900000000001 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466923+0000", 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00033036300000000002 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79737 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.325531835, 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011808300000000001, 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79737 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.839 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455334+0000", 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455333+0000", 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455339+0000", 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7570000000000001e-06 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455355+0000", 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.632e-05 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455368+0000", 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2588999999999999e-05 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455948+0000", 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058065099999999995 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455954+0000", 2024-09-13T04:59:31.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.466957+0000", 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011003361 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467033+0000", 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5644999999999998e-05 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467142+0000", 2024-09-13T04:59:31.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109527 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79757 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 393216~38443] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.325486523000002, 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013373497999999999, 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79757 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455378+0000", 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455384+0000", 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6200000000000004e-06 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455400+0000", 2024-09-13T04:59:31.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5973000000000001e-05 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455412+0000", 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2689e-05 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455979+0000", 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00056651800000000001 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:31.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455984+0000", 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.49e-06 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467237+0000", 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011252376999999999 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467453+0000", 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000216338 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468753+0000", 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012995459999999999 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79776 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.325441677000001, 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013932595000000001, 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.849 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79776 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455423+0000", 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455429+0000", 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455446+0000", 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6877000000000001e-05 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455459+0000", 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2866e-05 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456009+0000", 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000550838 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456015+0000", 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6060000000000002e-06 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468961+0000", 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012946301 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469181+0000", 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000220075 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469357+0000", 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000175279 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.854 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:79796 3.0 e60/24 3:003e46ae:::rbd_data.112acc4fa3fd.0000000000000000:head v 60'366, mlcod=58'364)", 2024-09-13T04:59:31.855 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:31.855 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.290109000000001, 2024-09-13T04:59:31.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025297777, 2024-09-13T04:59:31.855 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.855 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.855 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490755+0000", 2024-09-13T04:59:31.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494145+0000", 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033899300000000002 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494187+0000", 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1950999999999999e-05 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494201+0000", 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4134000000000001e-05 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494267+0000", 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5815999999999995e-05 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494296+0000", 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8597000000000001e-05 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516017+0000", 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021721285 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516054+0000", 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7389999999999999e-05 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.859 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79853 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.237128814000002, 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043236985999999998, 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79853 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543734+0000", 2024-09-13T04:59:31.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548313+0000", 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045782449999999999 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548339+0000", 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6948999999999999e-05 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548356+0000", 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6160000000000001e-05 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548459+0000", 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000103453 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548718+0000", 2024-09-13T04:59:31.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025891599999999999 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.549442+0000", 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00072383699999999996 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586544+0000", 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037102028000000002 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586859+0000", 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000315237 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586924+0000", 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5315999999999996e-05 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586974+0000", 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9481000000000003e-05 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81749 3.b 3:d52afe2d:::rbd_data.112a86dc1e25.000000000000000d:head [stat,delete] snapc 18=[18] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.708661724999999, 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010905319, 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.866 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81749 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072204+0000", 2024-09-13T04:59:31.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072201+0000", 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072215+0000", 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3859000000000001e-05 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072244+0000", 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8612999999999999e-05 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.072265+0000", 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.0893000000000001e-05 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.073443+0000", 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011780359999999999 2024-09-13T04:59:31.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.073753+0000", 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00031037700000000003 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.074309+0000", 2024-09-13T04:59:31.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00055601200000000004 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.076960+0000", 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.002650856 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.082880+0000", 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.005919901 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.083013+0000", 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000133168 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.083109+0000", 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6163999999999996e-05 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.872 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81735 3.8 e64/24 3:148b359f:::rbd_data.112a86dc1e25.0000000000000006:head v 64'139, mlcod=64'137)", 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.703145505000002, 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011631831, 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.873 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077720+0000", 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.077718+0000", 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078104+0000", 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000385852 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078144+0000", 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9858999999999999e-05 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078163+0000", 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8666999999999999e-05 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078282+0000", 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000119784 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.078334+0000", 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1712000000000002e-05 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.089290+0000", 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.010955515000000001 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.089352+0000", 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2389000000000002e-05 2024-09-13T04:59:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81782 3.c e64/24 3:33317f5e:::rbd_data.112a86dc1e25.000000000000000b:head v 64'410, mlcod=63'408)", 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.685813607999997, 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012792019999999999, 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095051+0000", 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:31.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095635+0000", 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058456600000000001 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095660+0000", 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4374000000000001e-05 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095670+0000", 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0631999999999999e-05 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095743+0000", 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3015000000000005e-05 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095775+0000", 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2092999999999999e-05 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107755+0000", 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011979949 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107844+0000", 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8858999999999997e-05 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81874 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [sparse-read 0~2423869] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.513148043999998, 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015773423000000002, 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81874 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267715+0000", 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:31.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267729+0000", 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4399999999999999e-05 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267761+0000", 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2112e-05 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267776+0000", 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4823e-05 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267903+0000", 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000127014 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.268135+0000", 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023146899999999999 2024-09-13T04:59:31.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.283491+0000", 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015356491 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:96142 3.0 e84/24 3:05f91f13:::rbd_data.112a487824ce.0000000000000000:head v 84'514, mlcod=82'512)", 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.5328065889999998, 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013444012999999999, 2024-09-13T04:59:31.886 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248058+0000", 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248437+0000", 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000378945 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248455+0000", 2024-09-13T04:59:31.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7791999999999998e-05 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248467+0000", 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1568e-05 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248575+0000", 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108637 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248605+0000", 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9471999999999998e-05 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.261461+0000", 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012856627000000001 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.261503+0000", 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656000000000002e-05 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.890 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:96241 3.b 3:d219a614:::rbd_data.112a3f73279b.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3069572~54962] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 9.2560558850000003, 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014882869999999999, 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 96241 2024-09-13T04:59:31.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524806+0000", 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525284+0000", 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047714799999999999 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525320+0000", 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6219e-05 2024-09-13T04:59:31.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525337+0000", 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7183000000000001e-05 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525421+0000", 2024-09-13T04:59:31.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3944999999999997e-05 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525613+0000", 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019242099999999999 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525898+0000", 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028419399999999998 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:31.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.533236+0000", 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073389190000000002 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539592+0000", 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.006355688 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539646+0000", 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4360000000000001e-05 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:31.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:31.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539693+0000", 2024-09-13T04:59:31.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6205999999999998e-05 2024-09-13T04:59:31.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.897 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.897 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:31.897 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:31.897 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:31.899 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:31.894+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:31.899 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:32.000 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:31.994+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:32.036 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:32.036 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:32.051 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:32.101 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:32.094+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:32.169 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:32.169 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:32.170 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:32.170 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:32.170 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:32.186 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:32.202 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:32.198+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:32.303 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:32.298+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:32.341 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone36 from 0x54dafb1 (overlap 0x37ef185) to 0x0 2024-09-13T04:59:32.358 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:32.358 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33208 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636728880999996, 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053460584999999998, 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33208 2024-09-13T04:59:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:59:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:59:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719001+0000", 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719003+0000", 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0389999999999999e-06 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719009+0000", 2024-09-13T04:59:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7490000000000004e-06 2024-09-13T04:59:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719013+0000", 2024-09-13T04:59:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550000000000001e-06 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772013+0000", 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053000688999999997 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772063+0000", 2024-09-13T04:59:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9774000000000002e-05 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772462+0000", 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039860600000000003 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636710210000004, 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053636234999999997, 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772488+0000", 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053447254999999999 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772562+0000", 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4276999999999994e-05 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772656+0000", 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3993999999999995e-05 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5898240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636684818000006, 2024-09-13T04:59:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054161726, 2024-09-13T04:59:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772681+0000", 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053622832000000002 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772751+0000", 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0166999999999994e-05 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773207+0000", 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045547300000000002 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5931008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636667367000001, 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054348847999999998, 2024-09-13T04:59:32.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773235+0000", 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054159997000000001 2024-09-13T04:59:32.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773316+0000", 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0727000000000001e-05 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773411+0000", 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5266999999999994e-05 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636650118999995, 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054777692000000003, 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:59:32.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773440+0000", 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054341649999999998 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773514+0000", 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4501999999999999e-05 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773858+0000", 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034316700000000002 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636627365999999, 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054958912999999998, 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:59:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773886+0000", 2024-09-13T04:59:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054770389000000003 2024-09-13T04:59:32.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773964+0000", 2024-09-13T04:59:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2024-09-13T04:59:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774062+0000", 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7059999999999996e-05 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636610415999996, 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055382103000000002, 2024-09-13T04:59:32.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:59:32.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:59:32.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:59:32.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774088+0000", 2024-09-13T04:59:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054950812000000002 2024-09-13T04:59:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774163+0000", 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5084000000000001e-05 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774502+0000", 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033872500000000001 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.398 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636588750000001, 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055600325999999999, 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:32.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:59:32.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774532+0000", 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055378563999999998 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774633+0000", 2024-09-13T04:59:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100519 2024-09-13T04:59:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774742+0000", 2024-09-13T04:59:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010891900000000001 2024-09-13T04:59:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6094848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636572514999997, 2024-09-13T04:59:32.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056713126000000003, 2024-09-13T04:59:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:59:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:32.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:32.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:59:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:59:32.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:59:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:59:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:59:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774769+0000", 2024-09-13T04:59:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055599061999999998 2024-09-13T04:59:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774843+0000", 2024-09-13T04:59:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3801999999999996e-05 2024-09-13T04:59:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775871+0000", 2024-09-13T04:59:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00102797 2024-09-13T04:59:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636556201999994, 2024-09-13T04:59:32.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056923247000000003, 2024-09-13T04:59:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:59:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:32.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:59:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:59:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:59:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:59:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:59:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:59:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:59:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775941+0000", 2024-09-13T04:59:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056755141000000002 2024-09-13T04:59:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775996+0000", 2024-09-13T04:59:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4580999999999998e-05 2024-09-13T04:59:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776097+0000", 2024-09-13T04:59:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101456 2024-09-13T04:59:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.636540201000003, 2024-09-13T04:59:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057407979999999997, 2024-09-13T04:59:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:59:32.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:59:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:59:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:59:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:59:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:59:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:59:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:59:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776138+0000", 2024-09-13T04:59:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056931362999999999 2024-09-13T04:59:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776293+0000", 2024-09-13T04:59:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015495600000000001 2024-09-13T04:59:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776598+0000", 2024-09-13T04:59:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030449200000000001 2024-09-13T04:59:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33298 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6012928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:59:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.625996357999995, 2024-09-13T04:59:32.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053286770999999997, 2024-09-13T04:59:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33298 2024-09-13T04:59:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:59:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729734+0000", 2024-09-13T04:59:32.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729733+0000", 2024-09-13T04:59:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729735+0000", 2024-09-13T04:59:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.875e-06 2024-09-13T04:59:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729741+0000", 2024-09-13T04:59:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.417e-06 2024-09-13T04:59:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729754+0000", 2024-09-13T04:59:32.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3811999999999999e-05 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.782831+0000", 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053076726999999997 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.782926+0000", 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5177e-05 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783020+0000", 2024-09-13T04:59:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4040000000000001e-05 2024-09-13T04:59:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.625951419000003, 2024-09-13T04:59:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053642026000000002, 2024-09-13T04:59:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.448 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:59:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:59:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783061+0000", 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053263004000000003 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783140+0000", 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9290999999999997e-05 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783421+0000", 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028007100000000001 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.625926934999995, 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053832005000000002, 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783450+0000", 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053635404999999997 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783532+0000", 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1310000000000006e-05 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783635+0000", 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010324599999999999 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.625910966999996, 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05399698, 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783666+0000", 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053834224999999999 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783745+0000", 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8542999999999997e-05 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783816+0000", 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279999999999995e-05 2024-09-13T04:59:32.462 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.625877696000003, 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054125706000000003, 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:59:32.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783842+0000", 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053971225999999997 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783915+0000", 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2657999999999995e-05 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783978+0000", 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3157000000000005e-05 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.467 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33307 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6176768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.624749131000002, 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053424891000000002, 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33307 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730981+0000", 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730980+0000", 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730983+0000", 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5359999999999999e-06 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730989+0000", 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5590000000000002e-06 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.730994+0000", 2024-09-13T04:59:32.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8040000000000001e-06 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784016+0000", 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053022133999999999 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784195+0000", 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017906399999999999 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784406+0000", 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021028099999999999 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.624714163999997, 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053864002000000001, 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:32.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784436+0000", 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053408069000000002 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784643+0000", 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000207171 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784880+0000", 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023680700000000001 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.476 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.624681735999999, 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054149968, 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:59:32.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784958+0000", 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053896985000000001 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785046+0000", 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8321999999999999e-05 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785198+0000", 2024-09-13T04:59:32.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015185 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.624020103000007, 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053689033999999997, 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:32.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:59:32.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785224+0000", 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053491874000000002 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:32.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785304+0000", 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9825000000000002e-05 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785399+0000", 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4837e-05 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:32.485 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:32.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:32.487 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:32.489 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:32.482+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:32.588 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:32.586+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:32.620 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-13T04:59:32.628 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:32.628 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:32.628 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:32.628 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:32.646 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:32.651 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:32.658 INFO:teuthology.orchestra.run.smithi001.stdout:5528 writesame 0xa53dfcf thru 0xa54b09c data_size 0x6(0xd0ce bytes) 2024-09-13T04:59:32.660 INFO:teuthology.orchestra.run.smithi001.stdout:5529 trunc from 0xa54b09d to 0x9203080 2024-09-13T04:59:32.668 INFO:teuthology.orchestra.run.smithi001.stdout:5530 write 0x365ae39 thru 0x3660703 (0x58cb bytes) 2024-09-13T04:59:32.671 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:32.672 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:32.689 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:32.686+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:32.690 INFO:teuthology.orchestra.run.smithi001.stdout:5531 trunc from 0x9203080 to 0x36391ae 2024-09-13T04:59:32.691 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:32.708 INFO:teuthology.orchestra.run.smithi001.stdout:5532 write 0x272d1cb thru 0x273c662 (0xf498 bytes) 2024-09-13T04:59:32.710 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:32.710 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:32.729 INFO:teuthology.orchestra.run.smithi001.stdout:5533 write 0x120c0f1 thru 0x121897b (0xc88b bytes) 2024-09-13T04:59:32.732 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:32.733 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:32.740 INFO:teuthology.orchestra.run.smithi001.stdout:5534 read 0x3009bd8 thru 0x3010955 (0x6d7e bytes) 2024-09-13T04:59:32.753 INFO:teuthology.orchestra.run.smithi001.stdout:5535 writesame 0x494d64b thru 0x4952bfd data_size 0x3(0x55b3 bytes) 2024-09-13T04:59:32.753 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:32.753 INFO:teuthology.orchestra.run.smithi001.stdout:5541 trunc from 0x4952bfe to 0x5c2f2a2 2024-09-13T04:59:32.756 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:32.757 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:32.774 INFO:teuthology.orchestra.run.smithi001.stdout:5543 trunc from 0x5c2f2a2 to 0xdb025df 2024-09-13T04:59:32.778 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:32.778 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:32.781 INFO:teuthology.orchestra.run.smithi001.stdout:5545 writesame 0x6b911e6 thru 0x6b9a013 data_size 0x1(0x8e2e bytes) 2024-09-13T04:59:32.790 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:32.786+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:32.792 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:32.792 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:32.799 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:32.805 INFO:teuthology.orchestra.run.smithi001.stdout:5547 read 0x778df5 thru 0x782668 (0x9874 bytes) 2024-09-13T04:59:32.807 INFO:teuthology.orchestra.run.smithi001.stdout:5548 write 0x7c0f606 thru 0x7c1a9dd (0xb3d8 bytes) 2024-09-13T04:59:32.809 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:32.820 INFO:teuthology.orchestra.run.smithi001.stdout:5550 writesame 0x628dd6a thru 0x629672c data_size 0x1(0x89c3 bytes) 2024-09-13T04:59:32.843 INFO:teuthology.orchestra.run.smithi001.stdout:5551 read 0xb2286ae thru 0xb23349d (0xadf0 bytes) 2024-09-13T04:59:32.846 INFO:teuthology.orchestra.run.smithi001.stdout:5553 trunc from 0xdb025df to 0xbf0758f 2024-09-13T04:59:32.856 INFO:teuthology.orchestra.run.smithi001.stdout:5554 write 0x1efc59a thru 0x1f04306 (0x7d6d bytes) 2024-09-13T04:59:32.866 INFO:teuthology.orchestra.run.smithi001.stdout:5556 write 0x8952fcd thru 0x895b3d2 (0x8406 bytes) 2024-09-13T04:59:32.876 INFO:teuthology.orchestra.run.smithi001.stdout:5557 punch from 0x904cd76 to 0x9057d79, (0xb003 bytes) 2024-09-13T04:59:32.876 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:32.876 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:32.876 INFO:teuthology.orchestra.run.smithi001.stdout:5560 read 0xbb91c9b thru 0xbb9f9ab (0xdd11 bytes) 2024-09-13T04:59:32.877 INFO:teuthology.orchestra.run.smithi001.stdout:5561 write 0xc20ae6 thru 0xc26927 (0x5e42 bytes) 2024-09-13T04:59:32.891 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:32.886+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:32.898 INFO:teuthology.orchestra.run.smithi001.stdout:5562 write 0xe8934bd thru 0xe89adf8 (0x793c bytes) 2024-09-13T04:59:32.918 INFO:teuthology.orchestra.run.smithi001.stdout:5564 trunc from 0xe89adf9 to 0x664192e 2024-09-13T04:59:32.938 INFO:teuthology.orchestra.run.smithi001.stdout:5565 write 0x46c5607 thru 0x46c5e32 (0x82c bytes) 2024-09-13T04:59:32.945 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:32.946 INFO:teuthology.orchestra.run.smithi001.stdout:5570 read 0x49f09bd thru 0x49f32e8 (0x292c bytes) 2024-09-13T04:59:32.946 INFO:teuthology.orchestra.run.smithi001.stdout:5571 punch from 0x44a87f0 to 0x44a9037, (0x847 bytes) 2024-09-13T04:59:32.946 INFO:teuthology.orchestra.run.smithi001.stdout:5572 read 0x54d2e45 thru 0x54da5f1 (0x77ad bytes) 2024-09-13T04:59:32.946 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:32.946 INFO:teuthology.orchestra.run.smithi001.stdout:5575 punch from 0x200c1da to 0x201801c, (0xbe42 bytes) 2024-09-13T04:59:32.946 INFO:teuthology.orchestra.run.smithi001.stdout:5576 punch from 0x4aa9ef9 to 0x4ab2e85, (0x8f8c bytes) 2024-09-13T04:59:32.952 INFO:teuthology.orchestra.run.smithi001.stdout:5579 trunc from 0x664192e to 0x3589cc5 2024-09-13T04:59:32.962 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:32.962 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:32.977 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:32.989 INFO:teuthology.orchestra.run.smithi001.stdout:5581 write 0x184d1bb thru 0x185b729 (0xe56f bytes) 2024-09-13T04:59:32.991 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:32.986+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:33.007 INFO:teuthology.orchestra.run.smithi001.stdout:5583 write 0x12f15af thru 0x12fe59e (0xcff0 bytes) 2024-09-13T04:59:33.019 INFO:teuthology.orchestra.run.smithi001.stdout:5584 write 0x224504c thru 0x224adb9 (0x5d6e bytes) 2024-09-13T04:59:33.027 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.051 INFO:teuthology.orchestra.run.smithi001.stdout:5586 write 0xd2fd2f3 thru 0xd2fe1fe (0xf0c bytes) 2024-09-13T04:59:33.058 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.058 INFO:teuthology.orchestra.run.smithi001.stdout:5589 write 0x4a045cc thru 0x4a0f7fe (0xb233 bytes) 2024-09-13T04:59:33.069 INFO:teuthology.orchestra.run.smithi001.stdout:5590 write 0xc43c394 thru 0xc44abef (0xe85c bytes) 2024-09-13T04:59:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:5592 write 0x97b0b5c thru 0x97b9b94 (0x9039 bytes) 2024-09-13T04:59:33.092 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:33.090+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:33.094 INFO:teuthology.orchestra.run.smithi001.stdout:5593 read 0xd1b692d thru 0xd1be4b7 (0x7b8b bytes) 2024-09-13T04:59:33.094 INFO:teuthology.orchestra.run.smithi001.stdout:5594 punch from 0x6afc3d9 to 0x6aff31b, (0x2f42 bytes) 2024-09-13T04:59:33.094 INFO:teuthology.orchestra.run.smithi001.stdout:5595 write 0xbe6f662 thru 0xbe72b5b (0x34fa bytes) 2024-09-13T04:59:33.104 INFO:teuthology.orchestra.run.smithi001.stdout:5596 write 0xc289ed0 thru 0xc28d6b0 (0x37e1 bytes) 2024-09-13T04:59:33.112 INFO:teuthology.orchestra.run.smithi001.stdout:5597 read 0x2c1a362 thru 0x2c21cea (0x7989 bytes) 2024-09-13T04:59:33.114 INFO:teuthology.orchestra.run.smithi001.stdout:5598 write 0x96d835d thru 0x96dd0c4 (0x4d68 bytes) 2024-09-13T04:59:33.116 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:33.117 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:33.122 INFO:teuthology.orchestra.run.smithi001.stdout:5600 read 0xd27f91b thru 0xd28c63d (0xcd23 bytes) 2024-09-13T04:59:33.122 INFO:teuthology.orchestra.run.smithi001.stdout:5601 write 0xa96275f thru 0xa96c08b (0x992d bytes) 2024-09-13T04:59:33.131 INFO:teuthology.orchestra.run.smithi001.stdout:5603 write 0x66da3f9 thru 0x66dd4af (0x30b7 bytes) 2024-09-13T04:59:33.134 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:33.137 INFO:teuthology.orchestra.run.smithi001.stdout:5604 write 0x25bf7d thru 0x26bed9 (0xff5d bytes) 2024-09-13T04:59:33.163 INFO:teuthology.orchestra.run.smithi001.stdout:5605 trunc from 0xd2fe1ff to 0x8f82577 2024-09-13T04:59:33.181 INFO:teuthology.orchestra.run.smithi001.stdout:5606 write 0x2dba13f thru 0x2dbb519 (0x13db bytes) 2024-09-13T04:59:33.191 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:33.192 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:33.192 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T04:59:33.192 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-13T04:59:33.192 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T04:59:33.194 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:33.190+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:33.198 INFO:teuthology.orchestra.run.smithi001.stdout:5608 read 0x1db0866 thru 0x1db181e (0xfb9 bytes) 2024-09-13T04:59:33.202 INFO:teuthology.orchestra.run.smithi001.stdout:5609 trunc from 0x8f82577 to 0xa9e7b07 2024-09-13T04:59:33.222 INFO:teuthology.orchestra.run.smithi001.stdout:5610 writesame 0xcf4b5b1 thru 0xcf4bf43 data_size 0x3(0x993 bytes) 2024-09-13T04:59:33.234 INFO:teuthology.orchestra.run.smithi001.stdout:5611 trunc from 0xcf4bf44 to 0x685c8bd 2024-09-13T04:59:33.247 INFO:teuthology.orchestra.run.smithi001.stdout:5612 trunc from 0x685c8bd to 0x3b63ca9 2024-09-13T04:59:33.276 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:33.276 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:33.282 INFO:teuthology.orchestra.run.smithi001.stdout:5613 write 0x9eb9558 thru 0x9ec46b1 (0xb15a bytes) 2024-09-13T04:59:33.287 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:59:33.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:33.295 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:33.290+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:33.301 INFO:teuthology.orchestra.run.smithi001.stdout:5614 trunc from 0x9ec46b2 to 0x44529ca 2024-09-13T04:59:33.322 INFO:teuthology.orchestra.run.smithi001.stdout:5615 read 0x2a67e77 thru 0x2a6c71a (0x48a4 bytes) 2024-09-13T04:59:33.322 INFO:teuthology.orchestra.run.smithi001.stdout:5617 read 0x10f545f thru 0x10faccf (0x5871 bytes) 2024-09-13T04:59:33.336 INFO:teuthology.orchestra.run.smithi001.stdout:5619 write 0x6c25a1a thru 0x6c2f3d2 (0x99b9 bytes) 2024-09-13T04:59:33.342 INFO:teuthology.orchestra.run.smithi001.stdout:5620 read 0x2cb2130 thru 0x2cbdb5c (0xba2d bytes) 2024-09-13T04:59:33.343 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.364 INFO:teuthology.orchestra.run.smithi001.stdout:5623 writesame 0xe0b1fbe thru 0xe0b81d4 data_size 0x1(0x6217 bytes) 2024-09-13T04:59:33.365 INFO:teuthology.orchestra.run.smithi001.stdout:5624 write 0xdba2be thru 0xdbc75e (0x24a1 bytes) 2024-09-13T04:59:33.374 INFO:teuthology.orchestra.run.smithi001.stdout:5626 trunc from 0xe0b81d5 to 0xe8cd4ec 2024-09-13T04:59:33.382 INFO:teuthology.orchestra.run.smithi001.stdout:5628 punch from 0x46c25db to 0x46c520c, (0x2c31 bytes) 2024-09-13T04:59:33.382 INFO:teuthology.orchestra.run.smithi001.stdout:5629 read 0x20adc94 thru 0x20b8b6c (0xaed9 bytes) 2024-09-13T04:59:33.383 INFO:teuthology.orchestra.run.smithi001.stdout:5630 punch from 0x56e891a to 0x56ee1de, (0x58c4 bytes) 2024-09-13T04:59:33.384 INFO:teuthology.orchestra.run.smithi001.stdout:5631 read 0x60a7560 thru 0x60b4151 (0xcbf2 bytes) 2024-09-13T04:59:33.384 INFO:teuthology.orchestra.run.smithi001.stdout:5632 read 0xc78696a thru 0xc78a083 (0x371a bytes) 2024-09-13T04:59:33.384 INFO:teuthology.orchestra.run.smithi001.stdout:5634 write 0x18b6fc7 thru 0x18bc258 (0x5292 bytes) 2024-09-13T04:59:33.388 INFO:teuthology.orchestra.run.smithi001.stdout:5636 punch from 0x54f227e to 0x54f4455, (0x21d7 bytes) 2024-09-13T04:59:33.389 INFO:teuthology.orchestra.run.smithi001.stdout:5637 read 0x977a2ea thru 0x9782ca9 (0x89c0 bytes) 2024-09-13T04:59:33.389 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.389 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.389 INFO:teuthology.orchestra.run.smithi001.stdout:5640 read 0xc9421d0 thru 0xc94b209 (0x903a bytes) 2024-09-13T04:59:33.389 INFO:teuthology.orchestra.run.smithi001.stdout:5641 punch from 0xac8db1b to 0xac9aecf, (0xd3b4 bytes) 2024-09-13T04:59:33.389 INFO:teuthology.orchestra.run.smithi001.stdout:5642 write 0x28fea35 thru 0x290031f (0x18eb bytes) 2024-09-13T04:59:33.394 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:33.390+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:33.399 INFO:teuthology.orchestra.run.smithi001.stdout:5643 read 0x41445ff thru 0x41490a8 (0x4aaa bytes) 2024-09-13T04:59:33.399 INFO:teuthology.orchestra.run.smithi001.stdout:5644 read 0x9c84704 thru 0x9c8cd21 (0x861e bytes) 2024-09-13T04:59:33.399 INFO:teuthology.orchestra.run.smithi001.stdout:5645 write 0xb8c59a6 thru 0xb8cdd14 (0x836f bytes) 2024-09-13T04:59:33.409 INFO:teuthology.orchestra.run.smithi001.stdout:5646 write 0x23a4f7b thru 0x23ac25a (0x72e0 bytes) 2024-09-13T04:59:33.417 INFO:teuthology.orchestra.run.smithi001.stdout:5648 writesame 0x5d3cdf9 thru 0x5d48487 data_size 0xd(0xb68f bytes) 2024-09-13T04:59:33.427 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.427 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.427 INFO:teuthology.orchestra.run.smithi001.stdout:5651 writesame 0x4fba1e5 thru 0x4fc6996 data_size 0x1(0xc7b2 bytes) 2024-09-13T04:59:33.435 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:33.435 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:33.451 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:33.459 INFO:teuthology.orchestra.run.smithi001.stdout:5652 writesame 0x37432eb thru 0x374cd0d data_size 0x7(0x9a23 bytes) 2024-09-13T04:59:33.470 INFO:teuthology.orchestra.run.smithi001.stdout:5654 write 0x1e23a95 thru 0x1e32cf3 (0xf25f bytes) 2024-09-13T04:59:33.481 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.481 INFO:teuthology.orchestra.run.smithi001.stdout:5656 write 0x31e7575 thru 0x31ee89f (0x732b bytes) 2024-09-13T04:59:33.489 INFO:teuthology.orchestra.run.smithi001.stdout:5658 writesame 0x25e08e thru 0x25e0e7 data_size 0x1(0x5a bytes) 2024-09-13T04:59:33.495 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.495 INFO:teuthology.orchestra.run.smithi001.stdout:5660 writesame 0x79b57d7 thru 0x79b8279 data_size 0x25(0x2aa3 bytes) 2024-09-13T04:59:33.495 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:33.490+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:33.503 INFO:teuthology.orchestra.run.smithi001.stdout:5661 write 0xd122703 thru 0xd12a9d6 (0x82d4 bytes) 2024-09-13T04:59:33.513 INFO:teuthology.orchestra.run.smithi001.stdout:5662 read 0x6ac4e thru 0x719c9 (0x6d7c bytes) 2024-09-13T04:59:33.514 INFO:teuthology.orchestra.run.smithi001.stdout:5663 write 0x6de3042 thru 0x6de4e56 (0x1e15 bytes) 2024-09-13T04:59:33.520 INFO:teuthology.orchestra.run.smithi001.stdout:5664 read 0x1d72d0a thru 0x1d784b3 (0x57aa bytes) 2024-09-13T04:59:33.522 INFO:teuthology.orchestra.run.smithi001.stdout:5665 writesame 0x18327aa thru 0x18338f6 data_size 0x1(0x114d bytes) 2024-09-13T04:59:33.534 INFO:teuthology.orchestra.run.smithi001.stdout:5666 writesame 0xdd8edf8 thru 0xdd927be data_size 0x1(0x39c7 bytes) 2024-09-13T04:59:33.552 INFO:teuthology.orchestra.run.smithi001.stdout:5667 writesame 0x7185539 thru 0x7189c35 data_size 0x1(0x46fd bytes) 2024-09-13T04:59:33.568 INFO:teuthology.orchestra.run.smithi001.stdout:5668 punch from 0x1e361ac to 0x1e37455, (0x12a9 bytes) 2024-09-13T04:59:33.568 INFO:teuthology.orchestra.run.smithi001.stdout:5669 read 0x4674be8 thru 0x46826bf (0xdad8 bytes) 2024-09-13T04:59:33.569 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.569 INFO:teuthology.orchestra.run.smithi001.stdout:5671 read 0xd3baece thru 0xd3c47dd (0x9910 bytes) 2024-09-13T04:59:33.569 INFO:teuthology.orchestra.run.smithi001.stdout:5672 write 0x5f4dd00 thru 0x5f59779 (0xba7a bytes) 2024-09-13T04:59:33.578 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:33.578 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:33.583 INFO:teuthology.orchestra.run.smithi001.stdout:5674 write 0x8cb3a5b thru 0x8cbc153 (0x86f9 bytes) 2024-09-13T04:59:33.591 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:33.594 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.596 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:33.590+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:33.608 INFO:teuthology.orchestra.run.smithi001.stdout:5677 trunc from 0xe8cd4ec to 0x47cc338 2024-09-13T04:59:33.653 INFO:teuthology.orchestra.run.smithi001.stdout:5678 write 0xb87af27 thru 0xb8874cd (0xc5a7 bytes) 2024-09-13T04:59:33.674 INFO:teuthology.orchestra.run.smithi001.stdout:5679 trunc from 0xb8874ce to 0x1a3483b 2024-09-13T04:59:33.697 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:33.694+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:5680 read 0xa8b7d7 thru 0xa8c374 (0xb9e bytes) 2024-09-13T04:59:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:5682 read 0x12951a thru 0x12a0d5 (0xbbc bytes) 2024-09-13T04:59:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:5683 read 0x1267319 thru 0x126b2c7 (0x3faf bytes) 2024-09-13T04:59:33.728 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:33.729 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:33.744 INFO:teuthology.orchestra.run.smithi001.stdout:5684 write 0xea1dedf thru 0xea28b58 (0xac7a bytes) 2024-09-13T04:59:33.747 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:33.754 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.754 INFO:teuthology.orchestra.run.smithi001.stdout:5686 write 0xe5206e4 thru 0xe52fe65 (0xf782 bytes) 2024-09-13T04:59:33.765 INFO:teuthology.orchestra.run.smithi001.stdout:5687 read 0x6cc03cf thru 0x6ccbbdc (0xb80e bytes) 2024-09-13T04:59:33.765 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:33.765 INFO:teuthology.orchestra.run.smithi001.stdout:5691 writesame 0xc2a71fb thru 0xc2b30fd data_size 0x1(0xbf03 bytes) 2024-09-13T04:59:33.798 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:33.794+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:33.823 INFO:teuthology.orchestra.run.smithi001.stdout:5692 read 0xc73f9b thru 0xc79f6b (0x5fd1 bytes) 2024-09-13T04:59:33.825 INFO:teuthology.orchestra.run.smithi001.stdout:5693 read 0x4cd1bea thru 0x4cd7935 (0x5d4c bytes) 2024-09-13T04:59:33.828 INFO:teuthology.orchestra.run.smithi001.stdout:5695 write 0x660efdb thru 0x661eb17 (0xfb3d bytes) 2024-09-13T04:59:33.843 INFO:teuthology.orchestra.run.smithi001.stdout:5697 write 0x849cb6d thru 0x84a31c8 (0x665c bytes) 2024-09-13T04:59:33.853 INFO:teuthology.orchestra.run.smithi001.stdout:5698 read 0xd4380a5 thru 0xd441d8b (0x9ce7 bytes) 2024-09-13T04:59:33.853 INFO:teuthology.orchestra.run.smithi001.stdout:5699 write 0x152b56c thru 0x153747e (0xbf13 bytes) 2024-09-13T04:59:33.868 INFO:teuthology.orchestra.run.smithi001.stdout:5700 punch from 0x1ede941 to 0x1eebf6c, (0xd62b bytes) 2024-09-13T04:59:33.879 INFO:teuthology.orchestra.run.smithi001.stdout:5701 trunc from 0xea28b59 to 0x8fb1bc9 2024-09-13T04:59:33.893 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:33.893 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:33.898 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:33.894+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:33.906 INFO:teuthology.orchestra.run.smithi001.stdout:5702 write 0x3b0378a thru 0x3b12a05 (0xf27c bytes) 2024-09-13T04:59:33.907 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:33.922 INFO:teuthology.orchestra.run.smithi001.stdout:5704 write 0x3efde64 thru 0x3f0613c (0x82d9 bytes) 2024-09-13T04:59:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:5705 writesame 0x1486e5d thru 0x14922ff data_size 0x1(0xb4a3 bytes) 2024-09-13T04:59:33.995 INFO:teuthology.orchestra.run.smithi001.stdout:5706 trunc from 0x8fb1bc9 to 0x747fd09 2024-09-13T04:59:34.000 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:33.994+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:34.025 INFO:teuthology.orchestra.run.smithi001.stdout:5707 write 0x5e55e7a thru 0x5e5c528 (0x66af bytes) 2024-09-13T04:59:34.034 INFO:teuthology.orchestra.run.smithi001.stdout:5708 read 0x205cd0e thru 0x205d673 (0x966 bytes) 2024-09-13T04:59:34.035 INFO:teuthology.orchestra.run.smithi001.stdout:5709 punch from 0x766ab7 to 0x774e9e, (0xe3e7 bytes) 2024-09-13T04:59:34.044 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:34.045 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:34.050 INFO:teuthology.orchestra.run.smithi001.stdout:5710 write 0x882801a thru 0x88335fe (0xb5e5 bytes) 2024-09-13T04:59:34.060 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:34.061 INFO:teuthology.orchestra.run.smithi001.stdout:5713 write 0x60dc986 thru 0x60df412 (0x2a8d bytes) 2024-09-13T04:59:34.067 INFO:teuthology.orchestra.run.smithi001.stdout:5714 punch from 0x374e677 to 0x3754323, (0x5cac bytes) 2024-09-13T04:59:34.067 INFO:teuthology.orchestra.run.smithi001.stdout:5715 read 0x1b7b222 thru 0x1b87414 (0xc1f3 bytes) 2024-09-13T04:59:34.067 INFO:teuthology.orchestra.run.smithi001.stdout:5716 writesame 0x552efa3 thru 0x552fd3a data_size 0x5(0xd98 bytes) 2024-09-13T04:59:34.073 INFO:teuthology.orchestra.run.smithi001.stdout:5717 read 0x37199a8 thru 0x37260c2 (0xc71b bytes) 2024-09-13T04:59:34.074 INFO:teuthology.orchestra.run.smithi001.stdout:5718 writesame 0x1ce3eef thru 0x1ce52e7 data_size 0x1(0x13f9 bytes) 2024-09-13T04:59:34.101 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:34.094+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:34.110 INFO:teuthology.orchestra.run.smithi001.stdout:5719 write 0xd7e5459 thru 0xd7e54cd (0x75 bytes) 2024-09-13T04:59:34.115 INFO:teuthology.orchestra.run.smithi001.stdout:5720 writesame 0xc8b8736 thru 0xc8c64ff data_size 0x3(0xddca bytes) 2024-09-13T04:59:34.145 INFO:teuthology.orchestra.run.smithi001.stdout:5721 write 0x75dcf57 thru 0x75e9b87 (0xcc31 bytes) 2024-09-13T04:59:34.159 INFO:teuthology.orchestra.run.smithi001.stdout:5722 write 0xc060e17 thru 0xc070b02 (0xfcec bytes) 2024-09-13T04:59:34.174 INFO:teuthology.orchestra.run.smithi001.stdout:5724 punch from 0xb4f039b to 0xb4fed39, (0xe99e bytes) 2024-09-13T04:59:34.176 INFO:teuthology.orchestra.run.smithi001.stdout:5725 trunc from 0xd7e54ce to 0xcc4ea36 2024-09-13T04:59:34.186 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:34.187 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:34.201 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T04:59:34.202 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:59:34.202 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:34.198+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:34.203 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:34.207 INFO:teuthology.orchestra.run.smithi001.stdout:5726 read 0x712dc9e thru 0x713342f (0x5792 bytes) 2024-09-13T04:59:34.207 INFO:teuthology.orchestra.run.smithi001.stdout:5727 punch from 0x9eefa40 to 0x9ef162b, (0x1beb bytes) 2024-09-13T04:59:34.207 INFO:teuthology.orchestra.run.smithi001.stdout:5728 write 0xc19b36b thru 0xc1a3b5c (0x87f2 bytes) 2024-09-13T04:59:34.215 INFO:teuthology.orchestra.run.smithi001.stdout:5729 read 0x52815f7 thru 0x52867c8 (0x51d2 bytes) 2024-09-13T04:59:34.224 INFO:teuthology.orchestra.run.smithi001.stdout:5730 write 0xd38a1cd thru 0xd3962fd (0xc131 bytes) 2024-09-13T04:59:34.231 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.231 INFO:teuthology.orchestra.run.smithi001.stdout:5732 read 0xc20d484 thru 0xc20e84d (0x13ca bytes) 2024-09-13T04:59:34.241 INFO:teuthology.orchestra.run.smithi001.stdout:5733 write 0xe89d54b thru 0xe8a4387 (0x6e3d bytes) 2024-09-13T04:59:34.246 INFO:teuthology.orchestra.run.smithi001.stdout:5734 read 0xc45b912 thru 0xc45caff (0x11ee bytes) 2024-09-13T04:59:34.246 INFO:teuthology.orchestra.run.smithi001.stdout:5738 writesame 0xc202571 thru 0xc20bb12 data_size 0x2(0x95a2 bytes) 2024-09-13T04:59:34.260 INFO:teuthology.orchestra.run.smithi001.stdout:5739 writesame 0xcff977b thru 0xcffe958 data_size 0x15(0x51de bytes) 2024-09-13T04:59:34.266 INFO:teuthology.orchestra.run.smithi001.stdout:5740 punch from 0xdb774c2 to 0xdb7e466, (0x6fa4 bytes) 2024-09-13T04:59:34.266 INFO:teuthology.orchestra.run.smithi001.stdout:5741 write 0xc463f85 thru 0xc466159 (0x21d5 bytes) 2024-09-13T04:59:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:5742 read 0xbcbf6f9 thru 0xbcc6484 (0x6d8c bytes) 2024-09-13T04:59:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:5744 write 0xde390de thru 0xde3d3d6 (0x42f9 bytes) 2024-09-13T04:59:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:5746 punch from 0x9469e65 to 0x946f26f, (0x540a bytes) 2024-09-13T04:59:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:5747 read 0x4930e81 thru 0x49339ca (0x2b4a bytes) 2024-09-13T04:59:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:5749 read 0x831818f thru 0x8323922 (0xb794 bytes) 2024-09-13T04:59:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:5750 read 0x1734b0b thru 0x1743d21 (0xf217 bytes) 2024-09-13T04:59:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:5753 writesame 0x5acfc95 thru 0x5adb6d5 data_size 0x1(0xba41 bytes) 2024-09-13T04:59:34.302 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:34.298+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:34.316 INFO:teuthology.orchestra.run.smithi001.stdout:5756 write 0x29cf90b thru 0x29d985f (0x9f55 bytes) 2024-09-13T04:59:34.331 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:34.331 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:34.332 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.332 INFO:teuthology.orchestra.run.smithi001.stdout:5760 punch from 0xae135b7 to 0xae14954, (0x139d bytes) 2024-09-13T04:59:34.333 INFO:teuthology.orchestra.run.smithi001.stdout:5763 punch from 0xca68ca2 to 0xca6e8ac, (0x5c0a bytes) 2024-09-13T04:59:34.333 INFO:teuthology.orchestra.run.smithi001.stdout:5764 read 0xbf83627 thru 0xbf91fc7 (0xe9a1 bytes) 2024-09-13T04:59:34.343 INFO:teuthology.orchestra.run.smithi001.stdout:5765 write 0xee61e11 thru 0xee6b27f (0x946f bytes) 2024-09-13T04:59:34.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:34.349 INFO:teuthology.orchestra.run.smithi001.stdout:5766 write 0x476f8cc thru 0x47781e9 (0x891e bytes) 2024-09-13T04:59:34.388 INFO:teuthology.orchestra.run.smithi001.stdout:5767 trunc from 0xee6b280 to 0x4bb62d1 2024-09-13T04:59:34.405 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:34.398+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:34.429 INFO:teuthology.orchestra.run.smithi001.stdout:5768 punch from 0x215ac5f to 0x2168721, (0xdac2 bytes) 2024-09-13T04:59:34.467 INFO:teuthology.orchestra.run.smithi001.stdout:5769 writesame 0xe734c05 thru 0xe743b70 data_size 0x9a(0xef6c bytes) 2024-09-13T04:59:34.467 INFO:teuthology.orchestra.run.smithi001.stdout:5770 read 0x9605017 thru 0x961303c (0xe026 bytes) 2024-09-13T04:59:34.467 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.468 INFO:teuthology.orchestra.run.smithi001.stdout:5773 writesame 0x2be8416 thru 0x2bec063 data_size 0x53(0x3c4e bytes) 2024-09-13T04:59:34.474 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:34.474 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:34.476 INFO:teuthology.orchestra.run.smithi001.stdout:5775 read 0xbfa60e8 thru 0xbfb3937 (0xd850 bytes) 2024-09-13T04:59:34.476 INFO:teuthology.orchestra.run.smithi001.stdout:5776 read 0xaf1bdb9 thru 0xaf2a7e6 (0xea2e bytes) 2024-09-13T04:59:34.476 INFO:teuthology.orchestra.run.smithi001.stdout:5777 writesame 0x72a7d10 thru 0x72b1f60 data_size 0x51(0xa251 bytes) 2024-09-13T04:59:34.484 INFO:teuthology.orchestra.run.smithi001.stdout:5778 read 0xda2b45c thru 0xda38c5b (0xd800 bytes) 2024-09-13T04:59:34.485 INFO:teuthology.orchestra.run.smithi001.stdout:5779 punch from 0x7d45f45 to 0x7d4cde1, (0x6e9c bytes) 2024-09-13T04:59:34.485 INFO:teuthology.orchestra.run.smithi001.stdout:5780 write 0x597410e thru 0x59754db (0x13ce bytes) 2024-09-13T04:59:34.490 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:34.493 INFO:teuthology.orchestra.run.smithi001.stdout:5781 writesame 0xc098009 thru 0xc0a5437 data_size 0x1(0xd42f bytes) 2024-09-13T04:59:34.537 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:34.534+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:34.596 INFO:teuthology.orchestra.run.smithi001.stdout:5782 writesame 0xe8a7d5e thru 0xe8af891 data_size 0x14(0x7b34 bytes) 2024-09-13T04:59:34.596 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.596 INFO:teuthology.orchestra.run.smithi001.stdout:5785 read 0x23b1914 thru 0x23c0ecc (0xf5b9 bytes) 2024-09-13T04:59:34.597 INFO:teuthology.orchestra.run.smithi001.stdout:5786 write 0xca480fd thru 0xca57232 (0xf136 bytes) 2024-09-13T04:59:34.605 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:34.602+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:34.608 INFO:teuthology.orchestra.run.smithi001.stdout:5787 write 0xc90c707 thru 0xc91ba85 (0xf37f bytes) 2024-09-13T04:59:34.618 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:34.618 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:34.620 INFO:teuthology.orchestra.run.smithi001.stdout:5788 trunc from 0xe8af892 to 0xe899818 2024-09-13T04:59:34.634 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:34.639 INFO:teuthology.orchestra.run.smithi001.stdout:5789 read 0xcf11a08 thru 0xcf15ac8 (0x40c1 bytes) 2024-09-13T04:59:34.639 INFO:teuthology.orchestra.run.smithi001.stdout:5790 read 0x7ad1f02 thru 0x7adb58a (0x9689 bytes) 2024-09-13T04:59:34.639 INFO:teuthology.orchestra.run.smithi001.stdout:5791 read 0x125559c thru 0x1263242 (0xdca7 bytes) 2024-09-13T04:59:34.665 INFO:teuthology.orchestra.run.smithi001.stdout:5792 trunc from 0xe899818 to 0x56dd9ce 2024-09-13T04:59:34.690 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.703 INFO:teuthology.orchestra.run.smithi001.stdout:5794 write 0x5ee3ab2 thru 0x5eed4fb (0x9a4a bytes) 2024-09-13T04:59:34.710 INFO:teuthology.orchestra.run.smithi001.stdout:5796 write 0x30edc01 thru 0x30fb918 (0xdd18 bytes) 2024-09-13T04:59:34.725 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:34.722+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:34.751 INFO:teuthology.orchestra.run.smithi001.stdout:5797 write 0xb9fa14d thru 0xba0479c (0xa650 bytes) 2024-09-13T04:59:34.773 INFO:teuthology.orchestra.run.smithi001.stdout:5799 trunc from 0xba0479d to 0x691909f 2024-09-13T04:59:34.799 INFO:teuthology.orchestra.run.smithi001.stdout:5802 trunc from 0x691909f to 0x7088c13 2024-09-13T04:59:34.804 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:34.805 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:34.806 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:34.802+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:34.811 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.811 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.819 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:34.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:34.837 INFO:teuthology.orchestra.run.smithi001.stdout:5805 write 0xbdf3d79 thru 0xbe0167e (0xd906 bytes) 2024-09-13T04:59:34.844 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.844 INFO:teuthology.orchestra.run.smithi001.stdout:5807 read 0x732133a thru 0x732e139 (0xce00 bytes) 2024-09-13T04:59:34.844 INFO:teuthology.orchestra.run.smithi001.stdout:5808 write 0x441d1d4 thru 0x441ef7a (0x1da7 bytes) 2024-09-13T04:59:34.849 INFO:teuthology.orchestra.run.smithi001.stdout:5809 write 0x5d3cfd3 thru 0x5d44420 (0x744e bytes) 2024-09-13T04:59:34.857 INFO:teuthology.orchestra.run.smithi001.stdout:5811 punch from 0x6753f5b to 0x6762f9b, (0xf040 bytes) 2024-09-13T04:59:34.857 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.857 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:34.857 INFO:teuthology.orchestra.run.smithi001.stdout:5815 write 0x331d290 thru 0x3326399 (0x910a bytes) 2024-09-13T04:59:34.872 INFO:teuthology.orchestra.run.smithi001.stdout:5816 trunc from 0xbe0167f to 0x69a8ff1 2024-09-13T04:59:34.897 INFO:teuthology.orchestra.run.smithi001.stdout:5817 trunc from 0x69a8ff1 to 0xd47ad6a 2024-09-13T04:59:34.904 INFO:teuthology.orchestra.run.smithi001.stdout:5818 read 0x857c388 thru 0x858a104 (0xdd7d bytes) 2024-09-13T04:59:34.905 INFO:teuthology.orchestra.run.smithi001.stdout:5820 writesame 0x9e0eddb thru 0x9e15581 data_size 0x1d(0x67a7 bytes) 2024-09-13T04:59:34.908 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:34.906+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:34.912 INFO:teuthology.orchestra.run.smithi001.stdout:5822 punch from 0x3804a56 to 0x38103f0, (0xb99a bytes) 2024-09-13T04:59:34.912 INFO:teuthology.orchestra.run.smithi001.stdout:5823 writesame 0xb04f890 thru 0xb05f1b7 data_size 0x88(0xf928 bytes) 2024-09-13T04:59:34.921 INFO:teuthology.orchestra.run.smithi001.stdout:5825 punch from 0xccc29ba to 0xccc3f19, (0x155f bytes) 2024-09-13T04:59:34.934 INFO:teuthology.orchestra.run.smithi001.stdout:5828 trunc from 0xd47ad6a to 0x73239cf 2024-09-13T04:59:34.949 INFO:teuthology.orchestra.run.smithi001.stdout:5829 write 0x6a89bfc thru 0x6a8bd1f (0x2124 bytes) 2024-09-13T04:59:34.976 INFO:teuthology.orchestra.run.smithi001.stdout:5830 write 0xda1c5e3 thru 0xda1ecd8 (0x26f6 bytes) 2024-09-13T04:59:34.982 INFO:teuthology.orchestra.run.smithi001.stdout:5831 punch from 0xd0851a2 to 0xd0931b8, (0xe016 bytes) 2024-09-13T04:59:34.986 INFO:teuthology.orchestra.run.smithi001.stdout:5833 trunc from 0xda1ecd9 to 0xac57f56 2024-09-13T04:59:34.991 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:34.991 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:34.991 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:34.991 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:35.003 INFO:teuthology.orchestra.run.smithi001.stdout:5834 writesame 0x256cf94 thru 0x257a227 data_size 0x2(0xd294 bytes) 2024-09-13T04:59:35.007 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:35.008 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:35.002+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:35.010 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:35.024 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:35.024 INFO:teuthology.orchestra.run.smithi001.stdout:5837 punch from 0x3a67c47 to 0x3a6b85e, (0x3c17 bytes) 2024-09-13T04:59:35.024 INFO:teuthology.orchestra.run.smithi001.stdout:5838 writesame 0xc7e92d thru 0xc8d6ad data_size 0xd(0xed81 bytes) 2024-09-13T04:59:35.027 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:35.027 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:35.036 INFO:teuthology.orchestra.run.smithi001.stdout:5839 trunc from 0xac57f56 to 0x80a8587 2024-09-13T04:59:35.046 INFO:teuthology.orchestra.run.smithi001.stdout:5840 read 0x34bbf62 thru 0x34c4981 (0x8a20 bytes) 2024-09-13T04:59:35.046 INFO:teuthology.orchestra.run.smithi001.stdout:5841 read 0x4c7a73f thru 0x4c7ad86 (0x648 bytes) 2024-09-13T04:59:35.046 INFO:teuthology.orchestra.run.smithi001.stdout:5842 read 0x3a2d9dd thru 0x3a3ca0b (0xf02f bytes) 2024-09-13T04:59:35.046 INFO:teuthology.orchestra.run.smithi001.stdout:5843 clone 38 order 18 su 1024 sc 13 2024-09-13T04:59:35.050 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:35.050 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:35.069 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:35.069 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:35.087 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:35.105 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:35.106 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:35.108 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:35.102+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:35.123 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:35.123 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:35.169 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:35.170 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:35.170 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:35.170 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:35.170 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:35.183 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:35.209 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:35.202+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:35.310 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:35.306+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:35.351 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:35.351 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549709073000002, 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.352 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:35.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549674819000003, 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.357 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:35.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549643918000001, 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:35.361 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:35.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549592036999996, 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:35.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:35.370 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549560233999998, 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:35.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.374 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549526280999999, 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:35.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:35.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549478544999999, 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.379 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.380 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:35.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:35.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:35.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:35.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:35.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:35.385 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549438983000002, 2024-09-13T04:59:35.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:35.386 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.386 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.386 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:35.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:35.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:35.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.390 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.390 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:35.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:35.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549391288999999, 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.391 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.395 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549328205999998, 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.396 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549291772000004, 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:35.400 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:35.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:35.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549257849, 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:35.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.409 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549189635999994, 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:35.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:35.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:35.414 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549155929999998, 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:35.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:35.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:35.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:35.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.421 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.549105737999994, 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:35.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:35.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:35.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.548799704999993, 2024-09-13T04:59:35.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:35.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.548750502999994, 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.431 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:35.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:35.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.548702388999999, 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.436 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:35.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.548650911999999, 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.440 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:35.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:35.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.548615831000006, 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.445 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:35.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.449 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:35.450 INFO:teuthology.orchestra.run.smithi001.stdout:truncating image image_client.0-clone37 from 0x80a8587 (overlap 0x1a3483b) to 0x7dc8de 2024-09-13T04:59:35.451 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:35.451 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:35.446+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:35.551 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:35.546+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:35.605 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-13T04:59:35.608 INFO:teuthology.orchestra.run.smithi001.stdout:5844 read 0x6c52356 thru 0x6c5cf3f (0xabea bytes) 2024-09-13T04:59:35.610 INFO:teuthology.orchestra.run.smithi001.stdout:5845 punch from 0x36f33fc to 0x36f7ef8, (0x4afc bytes) 2024-09-13T04:59:35.615 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:35.615 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:35.616 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:35.616 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:35.630 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:35.633 INFO:teuthology.orchestra.run.smithi001.stdout:5847 writesame 0x98c373e thru 0x98c853c data_size 0x1(0x4dff bytes) 2024-09-13T04:59:35.633 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:35.644 INFO:teuthology.orchestra.run.smithi001.stdout:5850 trunc from 0x98c853d to 0xe94a51e 2024-09-13T04:59:35.657 INFO:teuthology.orchestra.run.smithi001.stdout:5852 trunc from 0xe94a51e to 0xeb53b5a 2024-09-13T04:59:35.668 INFO:teuthology.orchestra.run.smithi001.stdout:5853 read 0x98789d2 thru 0x98835d3 (0xac02 bytes) 2024-09-13T04:59:35.669 INFO:teuthology.orchestra.run.smithi001.stdout:5854 writesame 0x6708d9f thru 0x671359e data_size 0x80(0xa800 bytes) 2024-09-13T04:59:35.676 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:35.674+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:35.729 INFO:teuthology.orchestra.run.smithi001.stdout:5855 trunc from 0xeb53b5a to 0x188b787 2024-09-13T04:59:35.755 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:35.750+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:35.777 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:35.777 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:35.777 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:35.777 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:35.777 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:35.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:35.884 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:35.882+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:35.913 INFO:teuthology.orchestra.run.smithi001.stdout:5857 write 0x8bdeb97 thru 0x8bee622 (0xfa8c bytes) 2024-09-13T04:59:35.942 INFO:teuthology.orchestra.run.smithi001.stdout:5858 writesame 0xa92a207 thru 0xa92a2a1 data_size 0x1(0x9b bytes) 2024-09-13T04:59:35.942 INFO:teuthology.orchestra.run.smithi001.stdout:5861 write 0x53a5697 thru 0x53b4686 (0xeff0 bytes) 2024-09-13T04:59:35.956 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:35.954+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:35.961 INFO:teuthology.orchestra.run.smithi001.stdout:5862 punch from 0x7c931eb to 0x7c96ead, (0x3cc2 bytes) 2024-09-13T04:59:35.961 INFO:teuthology.orchestra.run.smithi001.stdout:5863 read 0x7a73f17 thru 0x7a76dd8 (0x2ec2 bytes) 2024-09-13T04:59:35.961 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:35.961 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:35.961 INFO:teuthology.orchestra.run.smithi001.stdout:5868 write 0x65cad72 thru 0x65d06df (0x596e bytes) 2024-09-13T04:59:35.977 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:35.977 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:35.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49885 3.b 3:d43de71f:::rbd_data.112a40bf6452.0000000000000003:f [sparse-read 0~1843200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 59.946586959000001, 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015118388, 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49885 2024-09-13T04:59:35.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028151+0000", 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028150+0000", 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028157+0000", 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5510000000000002e-06 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028174+0000", 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6631000000000001e-05 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.980 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028183+0000", 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6100000000000006e-06 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028308+0000", 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012532600000000001 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.028557+0000", 2024-09-13T04:59:35.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024922200000000001 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.043270+0000", 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014712578 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49972 3.b 3:d2a26831:::rbd_data.112ae3a54590.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 59.795321553000001, 2024-09-13T04:59:35.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021756569999999999, 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49972 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:35.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179417+0000", 2024-09-13T04:59:35.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.179414+0000", 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180536+0000", 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001122381 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180563+0000", 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6211999999999999e-05 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180582+0000", 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9630999999999999e-05 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180662+0000", 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9411000000000003e-05 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180947+0000", 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000285166 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181294+0000", 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00034697800000000001 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200928+0000", 2024-09-13T04:59:35.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.019634766000000001 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201089+0000", 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016092299999999999 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201142+0000", 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2855000000000001e-05 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201173+0000", 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1096999999999997e-05 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49973 3.b 3:d63ae98e:::rbd_data.112ae3a54590.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1835008~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 59.794128014000002, 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.026409885000000001, 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:35.989 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49973 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180610+0000", 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.180608+0000", 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181851+0000", 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012428630000000001 2024-09-13T04:59:35.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181894+0000", 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2710000000000003e-05 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.181909+0000", 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5092999999999999e-05 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.186892+0000", 2024-09-13T04:59:35.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0049824229999999997 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187123+0000", 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023144999999999999 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.187416+0000", 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00029303400000000001 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.200934+0000", 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013517714 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206897+0000", 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0059634880000000003 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.206972+0000", 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5165000000000003e-05 2024-09-13T04:59:35.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207020+0000", 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7821000000000002e-05 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:49975 3.b 3:d71a7fd7:::rbd_data.112ae3a54590.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1826816~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 59.792469253999997, 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025198755, 2024-09-13T04:59:35.995 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 49975 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182269+0000", 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.182267+0000", 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183414+0000", 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011469270000000001 2024-09-13T04:59:35.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183442+0000", 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8189000000000001e-05 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.183458+0000", 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6073999999999999e-05 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:35.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.195878+0000", 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.01241922 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196039+0000", 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000161058 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.196279+0000", 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240676 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:35.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.201295+0000", 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0050153999999999997 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207382+0000", 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0060871650000000003 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207432+0000", 2024-09-13T04:59:36.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9882000000000001e-05 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:36.207468+0000", 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6140000000000003e-05 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79757 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 393216~38443] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:36.001 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.51935916, 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013373497999999999, 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79757 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455379+0000", 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455378+0000", 2024-09-13T04:59:36.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455384+0000", 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6200000000000004e-06 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455400+0000", 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5973000000000001e-05 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455412+0000", 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2689e-05 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455979+0000", 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00056651800000000001 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455984+0000", 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.49e-06 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467237+0000", 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011252376999999999 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.467453+0000", 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000216338 2024-09-13T04:59:36.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468753+0000", 2024-09-13T04:59:36.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0012995459999999999 2024-09-13T04:59:36.007 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.007 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.007 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79776 3.b 3:d6ace43f:::rbd_data.112aaf1f27a8.0000000000000038:16 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.519314313999999, 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013932595000000001, 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.008 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79776 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455424+0000", 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455423+0000", 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455429+0000", 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6999999999999996e-06 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455446+0000", 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6877000000000001e-05 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.455459+0000", 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2866e-05 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456009+0000", 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000550838 2024-09-13T04:59:36.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting_for_map not empty", 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.456015+0000", 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6060000000000002e-06 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.468961+0000", 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012946301 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469181+0000", 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000220075 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.469357+0000", 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000175279 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:79796 3.0 e60/24 3:003e46ae:::rbd_data.112acc4fa3fd.0000000000000000:head v 60'366, mlcod=58'364)", 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:36.013 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.483981636999999, 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.025297777, 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490757+0000", 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.490755+0000", 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494145+0000", 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0033899300000000002 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494187+0000", 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1950999999999999e-05 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494201+0000", 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4134000000000001e-05 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494267+0000", 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5815999999999995e-05 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.494296+0000", 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8597000000000001e-05 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516017+0000", 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.021721285 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.516054+0000", 2024-09-13T04:59:36.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7389999999999999e-05 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:79853 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 41.431001451, 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043236985999999998, 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:36.018 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 79853 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543737+0000", 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.543734+0000", 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548313+0000", 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0045782449999999999 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548339+0000", 2024-09-13T04:59:36.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6948999999999999e-05 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548356+0000", 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6160000000000001e-05 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548459+0000", 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000103453 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.548718+0000", 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025891599999999999 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.549442+0000", 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00072383699999999996 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586544+0000", 2024-09-13T04:59:36.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037102028000000002 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586859+0000", 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000315237 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586924+0000", 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5315999999999996e-05 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.586974+0000", 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9481000000000003e-05 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:81782 3.c e64/24 3:33317f5e:::rbd_data.112a86dc1e25.000000000000000b:head v 64'410, mlcod=63'408)", 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.879686245000002, 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012792019999999999, 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.024 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095052+0000", 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095051+0000", 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095635+0000", 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00058456600000000001 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095660+0000", 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4374000000000001e-05 2024-09-13T04:59:36.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095670+0000", 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0631999999999999e-05 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095743+0000", 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3015000000000005e-05 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.095775+0000", 2024-09-13T04:59:36.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2092999999999999e-05 2024-09-13T04:59:36.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.028 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107755+0000", 2024-09-13T04:59:36.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011979949 2024-09-13T04:59:36.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.107844+0000", 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8858999999999997e-05 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:81874 3.b 3:d01622fc:::rbd_data.112acc4fa3fd.0000000000000002:head [sparse-read 0~2423869] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.707020681000003, 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015773423000000002, 2024-09-13T04:59:36.029 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.030 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:36.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:36.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:36.030 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.030 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 81874 2024-09-13T04:59:36.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267718+0000", 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267715+0000", 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267729+0000", 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4399999999999999e-05 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267761+0000", 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2112e-05 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267776+0000", 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4823e-05 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.267903+0000", 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000127014 2024-09-13T04:59:36.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.268135+0000", 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00023146899999999999 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:58.283491+0000", 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.015356491 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:96142 3.0 e84/24 3:05f91f13:::rbd_data.112a487824ce.0000000000000000:head v 84'514, mlcod=82'512)", 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.726679226, 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.013444012999999999, 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248059+0000", 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248058+0000", 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248437+0000", 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000378945 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248455+0000", 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7791999999999998e-05 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248467+0000", 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1568e-05 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248575+0000", 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000108637 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.248605+0000", 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9471999999999998e-05 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.261461+0000", 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.012856627000000001 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.261503+0000", 2024-09-13T04:59:36.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656000000000002e-05 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:96241 3.b 3:d219a614:::rbd_data.112a3f73279b.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write 3069572~54962] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 13.449928522, 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.014882869999999999, 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:36.040 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 96241 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524810+0000", 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.524806+0000", 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525284+0000", 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047714799999999999 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525320+0000", 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6219e-05 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525337+0000", 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7183000000000001e-05 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525421+0000", 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3944999999999997e-05 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525613+0000", 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019242099999999999 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 6", 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.525898+0000", 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00028419399999999998 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.533236+0000", 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0073389190000000002 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539592+0000", 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.006355688 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539646+0000", 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4360000000000001e-05 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:22.539693+0000", 2024-09-13T04:59:36.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6205999999999998e-05 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99147 3.8 e92/24 3:17747dba:::rbd_data.112afaf35b85.0000000000000004:head v 92'362, mlcod=92'360)", 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5924948219999999, 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092626132, 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382240+0000", 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384691+0000", 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024508730000000001 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384738+0000", 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7345999999999997e-05 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384760+0000", 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1330000000000001e-05 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384885+0000", 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012574299999999999 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384941+0000", 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5210000000000002e-05 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474807+0000", 2024-09-13T04:59:36.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089866434999999995 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474870+0000", 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2692000000000002e-05 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99154 3.c e92/24 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head v 92'558, mlcod=91'556)", 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.402635+0000", 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5721036120000003, 2024-09-13T04:59:36.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.050835169999999999, 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.402635+0000", 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.402635+0000", 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.402632+0000", 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409230+0000", 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0065975119999999998 2024-09-13T04:59:36.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409262+0000", 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2251e-05 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409287+0000", 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.5352999999999999e-05 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409531+0000", 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00024387700000000001 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409601+0000", 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9601e-05 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.453412+0000", 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.043811614999999998 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.453470+0000", 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7441000000000001e-05 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99148 3.8 e92/24 3:17747dba:::rbd_data.112afaf35b85.0000000000000004:head v 92'364, mlcod=92'360)", 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5691331239999999, 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.074491979999999999, 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.056 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405603+0000", 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405621+0000", 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7927e-05 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405657+0000", 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5463999999999998e-05 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405675+0000", 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7935e-05 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.408148+0000", 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024729909999999999 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.408388+0000", 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240292 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480055+0000", 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.071666647 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480097+0000", 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2617000000000002e-05 2024-09-13T04:59:36.060 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99156 3.0 e92/24 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head v 92'539, mlcod=91'537)", 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5653062129999999, 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.070695703999999998, 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409430+0000", 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412318+0000", 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028881380000000002 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412366+0000", 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7565000000000003e-05 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412389+0000", 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3345000000000001e-05 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412654+0000", 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000264942 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412710+0000", 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6044000000000001e-05 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480102+0000", 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.067391567999999999 2024-09-13T04:59:36.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480128+0000", 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6251e-05 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99149 3.8 e92/24 3:14f8ab77:::rbd_data.112afaf35b85.0000000000000002:head v 92'365, mlcod=92'360)", 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5648179949999999, 2024-09-13T04:59:36.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.070251837999999997, 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409919+0000", 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414671+0000", 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0047520030000000003 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414735+0000", 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4139000000000004e-05 2024-09-13T04:59:36.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414757+0000", 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2327000000000002e-05 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.420020+0000", 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0052631450000000003 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.420067+0000", 2024-09-13T04:59:36.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6789999999999998e-05 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480133+0000", 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060066348999999998 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480172+0000", 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8757000000000002e-05 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99155 3.c e92/24 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head v 92'560, mlcod=91'556)", 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.416864+0000", 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5578740570000003, 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.037042485999999999, 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.416864+0000", 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.416864+0000", 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.416863+0000", 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.416876+0000", 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3485e-05 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.416916+0000", 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9845000000000002e-05 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.416931+0000", 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5384e-05 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.450906+0000", 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.033974227000000003 2024-09-13T04:59:36.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.450943+0000", 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7413999999999998e-05 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.453850+0000", 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0029071940000000001 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.453907+0000", 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6546999999999999e-05 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99150 3.8 e92/24 3:14f8ab77:::rbd_data.112afaf35b85.0000000000000002:head v 92'367, mlcod=92'360)", 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.445669+0000", 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5290696270000002, 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.035695910999999997, 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.445669+0000", 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.445669+0000", 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.445668+0000", 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.445676+0000", 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2560000000000002e-06 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.445694+0000", 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8054999999999999e-05 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.445702+0000", 2024-09-13T04:59:36.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2369999999999992e-06 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474910+0000", 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.029207911 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474928+0000", 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8436000000000001e-05 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.481327+0000", 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0063982029999999999 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.481365+0000", 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7988000000000003e-05 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99157 3.0 e92/24 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head v 92'541, mlcod=91'537)", 2024-09-13T04:59:36.078 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.447362+0000", 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 3.5273766540000002, 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.033756874999999999, 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.447362+0000", 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.447362+0000", 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:36.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.447360+0000", 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.447374+0000", 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3616e-05 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.447399+0000", 2024-09-13T04:59:36.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4890999999999999e-05 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.447411+0000", 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1715999999999999e-05 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474952+0000", 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.027541247000000001 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474988+0000", 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6148e-05 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.481051+0000", 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0060632339999999998 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.481119+0000", 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7422999999999995e-05 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.082 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:36.083 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:36.083 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:36.085 INFO:teuthology.orchestra.run.smithi001.stdout:5869 writesame 0x2606105 thru 0x260e1a4 data_size 0x2a(0x80a0 bytes) 2024-09-13T04:59:36.085 INFO:teuthology.orchestra.run.smithi001.stdout:5870 read 0x912c072 thru 0x912f132 (0x30c1 bytes) 2024-09-13T04:59:36.085 INFO:teuthology.orchestra.run.smithi001.stdout:5872 punch from 0x99a590e to 0x99b11eb, (0xb8dd bytes) 2024-09-13T04:59:36.085 INFO:teuthology.orchestra.run.smithi001.stdout:5873 write 0x9b84ecd thru 0x9b87288 (0x23bc bytes) 2024-09-13T04:59:36.085 INFO:teuthology.orchestra.run.smithi001.stdout:5875 writesame 0x340aee9 thru 0x3414d1a data_size 0x1(0x9e32 bytes) 2024-09-13T04:59:36.085 INFO:teuthology.orchestra.run.smithi001.stdout:5877 trunc from 0xa92a2a2 to 0x510c67d 2024-09-13T04:59:36.085 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:36.078+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:36.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:36.088 INFO:teuthology.orchestra.run.smithi001.stdout:5878 writesame 0x3fa586 thru 0x400003 data_size 0x63(0x5a7e bytes) 2024-09-13T04:59:36.185 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:36.182+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:36.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:59:36.294 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:36.290+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108114 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264630+0000", 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032838947, 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032862290000000002, 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108114 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264630+0000", 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264630+0000", 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264630+0000", 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264632+0000", 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3910000000000001e-06 2024-09-13T04:59:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264637+0000", 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3800000000000002e-06 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264642+0000", 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8810000000000004e-06 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108115 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264647+0000", 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032822513999999997, 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032901312000000002, 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108115 2024-09-13T04:59:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264647+0000", 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264647+0000", 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264646+0000", 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264649+0000", 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.362e-06 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264654+0000", 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8780000000000003e-06 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264659+0000", 2024-09-13T04:59:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9350000000000002e-06 2024-09-13T04:59:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108116 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264663+0000", 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032805764000000001, 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032948397999999997, 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108116 2024-09-13T04:59:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264663+0000", 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264663+0000", 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264663+0000", 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264665+0000", 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2409999999999998e-06 2024-09-13T04:59:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264671+0000", 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6370000000000004e-06 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264675+0000", 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5480000000000003e-06 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108117 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264680+0000", 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032788579999999998, 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032989347000000002, 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108117 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264680+0000", 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264680+0000", 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264680+0000", 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264683+0000", 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3630000000000001e-06 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264688+0000", 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3349999999999997e-06 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264693+0000", 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3959999999999997e-06 2024-09-13T04:59:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108124 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264697+0000", 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032771712000000001, 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033006385999999999, 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108124 2024-09-13T04:59:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264697+0000", 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264697+0000", 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264697+0000", 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264699+0000", 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3800000000000001e-06 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264707+0000", 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5190000000000003e-06 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264712+0000", 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6450000000000004e-06 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108129 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264716+0000", 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032753163000000002, 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033026482000000003, 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108129 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264716+0000", 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264716+0000", 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264716+0000", 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264718+0000", 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264725+0000", 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4019999999999996e-06 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264730+0000", 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7079999999999996e-06 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108130 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264734+0000", 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032734829999999999, 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033062866000000003, 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108130 2024-09-13T04:59:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264734+0000", 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264734+0000", 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264734+0000", 2024-09-13T04:59:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264736+0000", 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.255e-06 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264744+0000", 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4930000000000001e-06 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264748+0000", 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8119999999999998e-06 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108131 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264752+0000", 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032716861999999999, 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033107872000000003, 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108131 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264752+0000", 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264752+0000", 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264752+0000", 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264760+0000", 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8460000000000008e-06 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264768+0000", 2024-09-13T04:59:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8460000000000008e-06 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264773+0000", 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2259999999999996e-06 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108132 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264778+0000", 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032691502999999997, 2024-09-13T04:59:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033116659, 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108132 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264778+0000", 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264778+0000", 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264777+0000", 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264779+0000", 2024-09-13T04:59:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.295e-06 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264786+0000", 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.002e-06 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264791+0000", 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7659999999999998e-06 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108133 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264795+0000", 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032673643000000002, 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033132902999999998, 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108133 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264795+0000", 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264795+0000", 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264795+0000", 2024-09-13T04:59:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264797+0000", 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264805+0000", 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2099999999999996e-06 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264809+0000", 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108159 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264813+0000", 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032655649000000002, 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033175247999999997, 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108159 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264813+0000", 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264813+0000", 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264813+0000", 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264815+0000", 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.215e-06 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264822+0000", 2024-09-13T04:59:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1130000000000003e-06 2024-09-13T04:59:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264827+0000", 2024-09-13T04:59:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:59:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108160 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264831+0000", 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032637821999999997, 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033215184000000002, 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108160 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264831+0000", 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264831+0000", 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264831+0000", 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264833+0000", 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2630000000000002e-06 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264839+0000", 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0039999999999996e-06 2024-09-13T04:59:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264843+0000", 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2309999999999998e-06 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108161 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264848+0000", 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032621326999999999, 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033231442999999999, 2024-09-13T04:59:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108161 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264848+0000", 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264848+0000", 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264847+0000", 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264850+0000", 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4679999999999999e-06 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264856+0000", 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7599999999999999e-06 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264860+0000", 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6079999999999998e-06 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108162 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264864+0000", 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032604888999999998, 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033255449999999999, 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108162 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264864+0000", 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264864+0000", 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264864+0000", 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264866+0000", 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3870000000000002e-06 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264872+0000", 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7450000000000001e-06 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264877+0000", 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6840000000000004e-06 2024-09-13T04:59:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108163 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264882+0000", 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032587374000000002, 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033308113, 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108163 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264882+0000", 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264882+0000", 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264881+0000", 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264884+0000", 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.216e-06 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264889+0000", 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8830000000000003e-06 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264894+0000", 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5430000000000004e-06 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108184 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264898+0000", 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032570954999999999, 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033360842000000002, 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108184 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264898+0000", 2024-09-13T04:59:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264898+0000", 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264898+0000", 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264900+0000", 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7099999999999999e-06 2024-09-13T04:59:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264906+0000", 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8730000000000004e-06 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264911+0000", 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3919999999999996e-06 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108185 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264915+0000", 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032554485000000001, 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033400056999999997, 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108185 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264915+0000", 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264915+0000", 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264914+0000", 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264916+0000", 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2220000000000001e-06 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264923+0000", 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0920000000000003e-06 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264927+0000", 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.685e-06 2024-09-13T04:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108186 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264932+0000", 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032537576999999998, 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033443385999999999, 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108186 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264932+0000", 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264932+0000", 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264931+0000", 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264934+0000", 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3779999999999999e-06 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264941+0000", 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3640000000000001e-06 2024-09-13T04:59:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264946+0000", 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9609999999999996e-06 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108065 3.f 3:f7f71ef4:::rbd_data.112afaf35b85.0000000000000001:29 [read 1466368~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264380+0000", 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.033089239999999999, 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034059072000000003, 2024-09-13T04:59:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108065 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264380+0000", 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264380+0000", 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264380+0000", 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264382+0000", 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2390000000000001e-06 2024-09-13T04:59:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264389+0000", 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4819999999999997e-06 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264394+0000", 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6609999999999999e-06 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.298315+0000", 2024-09-13T04:59:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033921267999999997 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.298402+0000", 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7097999999999999e-05 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108187 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264950+0000", 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032519259000000002, 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033557254000000002, 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108187 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264950+0000", 2024-09-13T04:59:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264950+0000", 2024-09-13T04:59:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264950+0000", 2024-09-13T04:59:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264951+0000", 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264959+0000", 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1600000000000001e-06 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264963+0000", 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8489999999999997e-06 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108066 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264399+0000", 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.033070140999999997, 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034161992000000002, 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108066 2024-09-13T04:59:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264399+0000", 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264399+0000", 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264399+0000", 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264401+0000", 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3420000000000002e-06 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264408+0000", 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1409999999999999e-06 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264413+0000", 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108188 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264968+0000", 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032501245999999998, 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033651561000000003, 2024-09-13T04:59:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108188 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264968+0000", 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264968+0000", 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264967+0000", 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264970+0000", 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.52e-06 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264977+0000", 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.956e-06 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264982+0000", 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0180000000000001e-06 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108067 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264417+0000", 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.033052246, 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034260747000000001, 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108067 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264417+0000", 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264417+0000", 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264416+0000", 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264419+0000", 2024-09-13T04:59:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2120000000000002e-06 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264424+0000", 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7180000000000001e-06 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264429+0000", 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3760000000000001e-06 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108206 3.f 3.5de6c22f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.273833+0000", 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.023636041, 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024904180000000001, 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108206 2024-09-13T04:59:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.273833+0000", 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.273833+0000", 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.273750+0000", 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999917 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.273842+0000", 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2087000000000001e-05 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.273858+0000", 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6220000000000001e-05 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.273868+0000", 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4120000000000007e-06 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108068 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264433+0000", 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.033036426000000001, 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034364123000000003, 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108068 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264433+0000", 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264433+0000", 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264432+0000", 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264435+0000", 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4370000000000001e-06 2024-09-13T04:59:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264441+0000", 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2419999999999997e-06 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264446+0000", 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5070000000000002e-06 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108089 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264449+0000", 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.033019734000000002, 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034405446999999999, 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108089 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264449+0000", 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264449+0000", 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264449+0000", 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264451+0000", 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4830000000000002e-06 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264457+0000", 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6559999999999997e-06 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264462+0000", 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5600000000000004e-06 2024-09-13T04:59:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108090 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264465+0000", 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.033003847000000003, 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034441280999999997, 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108090 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264465+0000", 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264465+0000", 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264465+0000", 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264467+0000", 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1560000000000002e-06 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264473+0000", 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.604e-06 2024-09-13T04:59:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264477+0000", 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6759999999999998e-06 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108091 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264486+0000", 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032983353, 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034470210000000001, 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108091 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264486+0000", 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264486+0000", 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264485+0000", 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264488+0000", 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.34e-06 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264494+0000", 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4119999999999998e-06 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264499+0000", 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4039999999999997e-06 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108092 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264503+0000", 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032966374999999999, 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034486203999999999, 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108092 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264503+0000", 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264503+0000", 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264502+0000", 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264505+0000", 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2680000000000001e-06 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264511+0000", 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1070000000000001e-06 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264516+0000", 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.853e-06 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108093 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264520+0000", 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032949516999999998, 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034501271, 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108093 2024-09-13T04:59:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264520+0000", 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264520+0000", 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264519+0000", 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264522+0000", 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3530000000000002e-06 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264527+0000", 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7610000000000004e-06 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264532+0000", 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0699999999999997e-06 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108104 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264536+0000", 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032932768000000001, 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034528855999999997, 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108104 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264536+0000", 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264536+0000", 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264536+0000", 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264543+0000", 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6880000000000002e-06 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264550+0000", 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3919999999999997e-06 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264555+0000", 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9459999999999997e-06 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108105 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264559+0000", 2024-09-13T04:59:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032909755999999998, 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034575174, 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108105 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264559+0000", 2024-09-13T04:59:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264559+0000", 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264559+0000", 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264561+0000", 2024-09-13T04:59:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4849999999999999e-06 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264569+0000", 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3830000000000003e-06 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264574+0000", 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9180000000000002e-06 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108106 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264578+0000", 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032891400000000001, 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034615101000000002, 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108106 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264578+0000", 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264578+0000", 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264577+0000", 2024-09-13T04:59:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264580+0000", 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1919999999999999e-06 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264586+0000", 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8759999999999999e-06 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264591+0000", 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9429999999999999e-06 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108107 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264595+0000", 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032873727999999998, 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034639102999999997, 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108107 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264595+0000", 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264595+0000", 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264595+0000", 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264597+0000", 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1689999999999999e-06 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264603+0000", 2024-09-13T04:59:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7570000000000001e-06 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264608+0000", 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0640000000000001e-06 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:108113 3.f 3.2f78efef (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.264612+0000", 2024-09-13T04:59:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.032856997999999998, 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034685753, 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 108113 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264612+0000", 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264612+0000", 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264612+0000", 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264614+0000", 2024-09-13T04:59:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4269999999999998e-06 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264621+0000", 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2339999999999997e-06 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.264626+0000", 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7979999999999996e-06 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.421 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:36.421 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 35 2024-09-13T04:59:36.421 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:36.422 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:36.423 INFO:teuthology.orchestra.run.smithi001.stdout:5879 read 0x2128fb8 thru 0x212fcf5 (0x6d3e bytes) 2024-09-13T04:59:36.423 INFO:teuthology.orchestra.run.smithi001.stdout:5880 read 0x25ebc66 thru 0x25f72e0 (0xb67b bytes) 2024-09-13T04:59:36.423 INFO:teuthology.orchestra.run.smithi001.stdout:5881 write 0xca436 thru 0xd98db (0xf4a6 bytes) 2024-09-13T04:59:36.423 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:36.418+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:36.523 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:36.518+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:36.585 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:36.585 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:36.585 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:36.585 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:36.585 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:36.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:36.645 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:36.638+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:36.726 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:36.722+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:36.741 INFO:teuthology.orchestra.run.smithi001.stdout:5882 write 0x6032b29 thru 0x603ed59 (0xc231 bytes) 2024-09-13T04:59:36.790 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:36.790 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:36.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:36.790 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:36.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.790 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068674443000006, 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053636234999999997, 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772488+0000", 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053447254999999999 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772562+0000", 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4276999999999994e-05 2024-09-13T04:59:36.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772656+0000", 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3993999999999995e-05 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5898240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068649050999994, 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054161726, 2024-09-13T04:59:36.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772681+0000", 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053622832000000002 2024-09-13T04:59:36.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772751+0000", 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0166999999999994e-05 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773207+0000", 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045547300000000002 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.799 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5931008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068631600000003, 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054348847999999998, 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773235+0000", 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054159997000000001 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773316+0000", 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0727000000000001e-05 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773411+0000", 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5266999999999994e-05 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068614351999997, 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054777692000000003, 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773440+0000", 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054341649999999998 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773514+0000", 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4501999999999999e-05 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773858+0000", 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034316700000000002 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068591599000001, 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054958912999999998, 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:59:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773886+0000", 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054770389000000003 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773964+0000", 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774062+0000", 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7059999999999996e-05 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068574648999999, 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055382103000000002, 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774088+0000", 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054950812000000002 2024-09-13T04:59:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774163+0000", 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5084000000000001e-05 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774502+0000", 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033872500000000001 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068552983000004, 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055600325999999999, 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:59:36.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774532+0000", 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055378563999999998 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774633+0000", 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100519 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774742+0000", 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010891900000000001 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6094848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068536748, 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056713126000000003, 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:36.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774769+0000", 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055599061999999998 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774843+0000", 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3801999999999996e-05 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775871+0000", 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00102797 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.826 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068520434999996, 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056923247000000003, 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:36.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775941+0000", 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056755141000000002 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775996+0000", 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4580999999999998e-05 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776097+0000", 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101456 2024-09-13T04:59:36.830 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068504434000005, 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057407979999999997, 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:59:36.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776138+0000", 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056931362999999999 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776293+0000", 2024-09-13T04:59:36.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015495600000000001 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776598+0000", 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030449200000000001 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.057915652000005, 2024-09-13T04:59:36.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053642026000000002, 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:59:36.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783061+0000", 2024-09-13T04:59:36.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053263004000000003 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783140+0000", 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9290999999999997e-05 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783421+0000", 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028007100000000001 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.057891167999998, 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053832005000000002, 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.840 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:36.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:59:36.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783450+0000", 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053635404999999997 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783532+0000", 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1310000000000006e-05 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783635+0000", 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010324599999999999 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.057875199999998, 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05399698, 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.844 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:36.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:59:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783666+0000", 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053834224999999999 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783745+0000", 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8542999999999997e-05 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783816+0000", 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279999999999995e-05 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.057841929000006, 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054125706000000003, 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:59:36.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783842+0000", 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053971225999999997 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783915+0000", 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2657999999999995e-05 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783978+0000", 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3157000000000005e-05 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.056678396999999, 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053864002000000001, 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.853 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:36.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784436+0000", 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053408069000000002 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784643+0000", 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000207171 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784880+0000", 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023680700000000001 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.056645969000002, 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054149968, 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784958+0000", 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053896985000000001 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785046+0000", 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8321999999999999e-05 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785198+0000", 2024-09-13T04:59:36.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015185 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.055984335999995, 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053689033999999997, 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785224+0000", 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053491874000000002 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785304+0000", 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9825000000000002e-05 2024-09-13T04:59:36.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785399+0000", 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4837e-05 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99151 3.f 3:f7f71ef4:::rbd_data.112afaf35b85.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.4056088789999999, 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053542113000000002, 2024-09-13T04:59:36.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99151 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382083+0000", 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384133+0000", 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002049871 2024-09-13T04:59:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384157+0000", 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3620000000000001e-05 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384170+0000", 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3152999999999999e-05 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384238+0000", 2024-09-13T04:59:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7883e-05 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384350+0000", 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112525 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384768+0000", 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041751899999999998 2024-09-13T04:59:36.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426232+0000", 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041464366000000002 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435554+0000", 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093217539999999998 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435589+0000", 2024-09-13T04:59:36.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4468000000000003e-05 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435627+0000", 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8892e-05 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99154 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.3924502829999996, 2024-09-13T04:59:36.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058608128000000002, 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99154 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395242+0000", 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396241+0000", 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00099964600000000008 2024-09-13T04:59:36.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396263+0000", 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1939000000000001e-05 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396276+0000", 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2758999999999999e-05 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396371+0000", 2024-09-13T04:59:36.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5233000000000001e-05 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396524+0000", 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015272899999999999 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396949+0000", 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042510200000000001 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.418937+0000", 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021988085000000001 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453775+0000", 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034838002999999999 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453813+0000", 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8180999999999999e-05 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453852+0000", 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8852e-05 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99155 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.3913883890000003, 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057964565000000003, 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99155 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396305+0000", 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396310+0000", 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0950000000000003e-06 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396318+0000", 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.191e-06 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396324+0000", 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.959e-06 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416450+0000", 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020125707999999999 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416531+0000", 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1093000000000004e-05 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416721+0000", 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019040800000000001 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.419351+0000", 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026298229999999999 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454195+0000", 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034844092 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454241+0000", 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5918000000000003e-05 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454270+0000", 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9391e-05 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:36.884 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:36.884 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:36.885 INFO:teuthology.orchestra.run.smithi001.stdout:5883 write 0xd727c5c thru 0xd7301d6 (0x857b bytes) 2024-09-13T04:59:36.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:36.886 INFO:teuthology.orchestra.run.smithi001.stdout:5884 punch from 0xa5cd6ff to 0xa5cfa9b, (0x239c bytes) 2024-09-13T04:59:36.886 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:36.886 INFO:teuthology.orchestra.run.smithi001.stdout:5887 punch from 0x423bb46 to 0x42438b7, (0x7d71 bytes) 2024-09-13T04:59:36.887 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:36.887 INFO:teuthology.orchestra.run.smithi001.stdout:5889 trunc from 0xd7301d7 to 0xd5e8388 2024-09-13T04:59:36.887 INFO:teuthology.orchestra.run.smithi001.stdout:5890 writesame 0xb4da3a0 thru 0xb4e564e data_size 0x1(0xb2af bytes) 2024-09-13T04:59:36.887 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:36.882+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:36.892 INFO:teuthology.orchestra.run.smithi001.stdout:5892 write 0xb963167 thru 0xb96d16d (0xa007 bytes) 2024-09-13T04:59:36.915 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:36.915 INFO:teuthology.orchestra.run.smithi001.stdout:5894 writesame 0x16f78a0 thru 0x16f9b74 data_size 0x1(0x22d5 bytes) 2024-09-13T04:59:36.925 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:36.925 INFO:teuthology.orchestra.run.smithi001.stdout:5897 read 0x92fdb2d thru 0x92fffb6 (0x248a bytes) 2024-09-13T04:59:36.925 INFO:teuthology.orchestra.run.smithi001.stdout:5899 writesame 0x7680015 thru 0x7686cc9 data_size 0x1(0x6cb5 bytes) 2024-09-13T04:59:36.950 INFO:teuthology.orchestra.run.smithi001.stdout:5900 write 0x56920a3 thru 0x5698173 (0x60d1 bytes) 2024-09-13T04:59:36.981 INFO:teuthology.orchestra.run.smithi001.stdout:5901 writesame 0xdcea10d thru 0xdceb950 data_size 0x2(0x1844 bytes) 2024-09-13T04:59:36.981 INFO:teuthology.orchestra.run.smithi001.stdout:5902 write 0x873b007 thru 0x87416a1 (0x669b bytes) 2024-09-13T04:59:36.986 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:36.982+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:37.003 INFO:teuthology.orchestra.run.smithi001.stdout:5903 read 0x1d5675 thru 0x1daf84 (0x5910 bytes) 2024-09-13T04:59:37.006 INFO:teuthology.orchestra.run.smithi001.stdout:5904 punch from 0x134316d to 0x1343e6c, (0xcff bytes) 2024-09-13T04:59:37.006 INFO:teuthology.orchestra.run.smithi001.stdout:5905 read 0xd3370fd thru 0xd342607 (0xb50b bytes) 2024-09-13T04:59:37.006 INFO:teuthology.orchestra.run.smithi001.stdout:5906 punch from 0x24ab4bc to 0x24afa53, (0x4597 bytes) 2024-09-13T04:59:37.006 INFO:teuthology.orchestra.run.smithi001.stdout:5907 write 0xf9bad4 thru 0xfa457f (0x8aac bytes) 2024-09-13T04:59:37.043 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:113537 3.10 3.3fa87670 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:37.041508+0000", 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0019990110000000002, 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002025701, 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113537 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041508+0000", 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041508+0000", 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041507+0000", 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041511+0000", 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.174e-06 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041518+0000", 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2370000000000003e-06 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041523+0000", 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1120000000000004e-06 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:113488 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 3623936~1024 out=1024b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:37.039070+0000", 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0044370850000000003, 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045363620000000004, 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113488 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039070+0000", 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039070+0000", 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039069+0000", 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039072+0000", 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0750000000000002e-06 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039079+0000", 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3890000000000002e-06 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039084+0000", 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1170000000000003e-06 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.043451+0000", 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043672049999999999 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.043529+0000", 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7837999999999996e-05 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:113492 3.10 3.3fa87670 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:37.039088+0000", 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0044190180000000003, 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045793070000000003, 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113492 2024-09-13T04:59:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039088+0000", 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039088+0000", 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039088+0000", 2024-09-13T04:59:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039090+0000", 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.39e-06 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039096+0000", 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8830000000000003e-06 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039101+0000", 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9019999999999998e-06 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:113494 3.10 3.3fa87670 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:37.039105+0000", 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0044023710000000004, 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046185879999999999, 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113494 2024-09-13T04:59:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039105+0000", 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039105+0000", 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039104+0000", 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039106+0000", 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.165e-06 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039112+0000", 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6250000000000004e-06 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039117+0000", 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8679999999999998e-06 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:113498 3.10 3.3fa87670 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:37.039121+0000", 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0043860339999999996, 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046633109999999998, 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113498 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039121+0000", 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039121+0000", 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039121+0000", 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039123+0000", 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.046e-06 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039128+0000", 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6149999999999996e-06 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.039133+0000", 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4469999999999999e-06 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:113535 3.10 3.3fa87670 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:37.041471+0000", 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0020356739999999999, 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023495550000000001, 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "queued for pg", 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 113535 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041471+0000", 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041471+0000", 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041470+0000", 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041476+0000", 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1920000000000002e-06 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041493+0000", 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6322000000000001e-05 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:37.041501+0000", 2024-09-13T04:59:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2024-09-13T04:59:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-13T04:59:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 6 2024-09-13T04:59:37.068 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:37.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:37.115 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:37.110+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:37.141 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:37.169 INFO:teuthology.orchestra.run.smithi001.stdout:5908 read 0x16d9255 thru 0x16dedf0 (0x5b9c bytes) 2024-09-13T04:59:37.171 INFO:teuthology.orchestra.run.smithi001.stdout:5909 read 0x1dafd2b thru 0x1db86bc (0x8992 bytes) 2024-09-13T04:59:37.171 INFO:teuthology.orchestra.run.smithi001.stdout:5911 writesame 0xcbd6718 thru 0xcbdc13b data_size 0x12(0x5a24 bytes) 2024-09-13T04:59:37.187 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:37.182+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:37.189 INFO:teuthology.orchestra.run.smithi001.stdout:5912 write 0x4f11b21 thru 0x4f1c1b4 (0xa694 bytes) 2024-09-13T04:59:37.206 INFO:teuthology.orchestra.run.smithi001.stdout:5913 read 0x61c920 thru 0x626ca2 (0xa383 bytes) 2024-09-13T04:59:37.208 INFO:teuthology.orchestra.run.smithi001.stdout:5915 punch from 0x337a506 to 0x33871ed, (0xcce7 bytes) 2024-09-13T04:59:37.208 INFO:teuthology.orchestra.run.smithi001.stdout:5916 read 0x69a442a thru 0x69af718 (0xb2ef bytes) 2024-09-13T04:59:37.208 INFO:teuthology.orchestra.run.smithi001.stdout:5917 read 0x9f98c9c thru 0x9f9b6f0 (0x2a55 bytes) 2024-09-13T04:59:37.208 INFO:teuthology.orchestra.run.smithi001.stdout:5919 writesame 0x2667495 thru 0x266fbfd data_size 0x5(0x8769 bytes) 2024-09-13T04:59:37.217 INFO:teuthology.orchestra.run.smithi001.stdout:5920 write 0xbcb33bb thru 0xbcb69a6 (0x35ec bytes) 2024-09-13T04:59:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:59:37.241 INFO:teuthology.orchestra.run.smithi001.stdout:5922 write 0x9223567 thru 0x9228990 (0x542a bytes) 2024-09-13T04:59:37.260 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.260 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.268 INFO:teuthology.orchestra.run.smithi001.stdout:5925 trunc from 0xdceb951 to 0x5df97d8 2024-09-13T04:59:37.273 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:37.273 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:37.291 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:37.286+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:37.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:37.344 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:37.346 INFO:teuthology.orchestra.run.smithi001.stdout:5926 write 0xde50e70 thru 0xde5d805 (0xc996 bytes) 2024-09-13T04:59:37.351 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:37.352 INFO:teuthology.orchestra.run.smithi001.stdout:5927 write 0x1f5d50c thru 0x1f63cb9 (0x67ae bytes) 2024-09-13T04:59:37.362 INFO:teuthology.orchestra.run.smithi001.stdout:5929 punch from 0x5bfbf7b to 0x5c08fd0, (0xd055 bytes) 2024-09-13T04:59:37.362 INFO:teuthology.orchestra.run.smithi001.stdout:5930 punch from 0x1184a01 to 0x118a171, (0x5770 bytes) 2024-09-13T04:59:37.362 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:37.362 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:37.363 INFO:teuthology.orchestra.run.smithi001.stdout:5931 read 0xa60d898 thru 0xa61ac85 (0xd3ee bytes) 2024-09-13T04:59:37.363 INFO:teuthology.orchestra.run.smithi001.stdout:5932 punch from 0x7054b90 to 0x705bb11, (0x6f81 bytes) 2024-09-13T04:59:37.369 INFO:teuthology.orchestra.run.smithi001.stdout:5933 write 0xeb176fe thru 0xeb1d958 (0x625b bytes) 2024-09-13T04:59:37.380 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:37.380 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:37.389 INFO:teuthology.orchestra.run.smithi001.stdout:5935 write 0x2d32de thru 0x2e3011 (0xfd34 bytes) 2024-09-13T04:59:37.390 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:37.386+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:37.395 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.395 INFO:teuthology.orchestra.run.smithi001.stdout:5937 write 0x342aaa2 thru 0x34318af (0x6e0e bytes) 2024-09-13T04:59:37.402 INFO:teuthology.orchestra.run.smithi001.stdout:5938 read 0x669d15d thru 0x66a27f2 (0x5696 bytes) 2024-09-13T04:59:37.403 INFO:teuthology.orchestra.run.smithi001.stdout:5939 read 0xe614eb4 thru 0xe61d35b (0x84a8 bytes) 2024-09-13T04:59:37.403 INFO:teuthology.orchestra.run.smithi001.stdout:5940 writesame 0x92a37b7 thru 0x92aa332 data_size 0x6(0x6b7c bytes) 2024-09-13T04:59:37.403 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:37.403 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:37.423 INFO:teuthology.orchestra.run.smithi001.stdout:5941 write 0x109e482 thru 0x109eea9 (0xa28 bytes) 2024-09-13T04:59:37.424 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:37.424 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:37.455 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:37.455 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:37.470 INFO:teuthology.orchestra.run.smithi001.stdout:5942 punch from 0x4579a7 to 0x463ef9, (0xc552 bytes) 2024-09-13T04:59:37.471 INFO:teuthology.orchestra.run.smithi001.stdout:5943 write 0xdd11d85 thru 0xdd1978d (0x7a09 bytes) 2024-09-13T04:59:37.473 INFO:teuthology.orchestra.run.smithi001.stdout:5944 writesame 0x90cf35a thru 0x90db564 data_size 0x5(0xc20b bytes) 2024-09-13T04:59:37.476 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:37.493 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:37.486+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:37.494 INFO:teuthology.orchestra.run.smithi001.stdout:5945 writesame 0xdea42a5 thru 0xdeaf847 data_size 0x1(0xb5a3 bytes) 2024-09-13T04:59:37.495 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:37.496 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:37.512 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:37.523 INFO:teuthology.orchestra.run.smithi001.stdout:5946 punch from 0x242ef77 to 0x242ffda, (0x1063 bytes) 2024-09-13T04:59:37.523 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.524 INFO:teuthology.orchestra.run.smithi001.stdout:5949 punch from 0x5a9cb13 to 0x5a9d4d9, (0x9c6 bytes) 2024-09-13T04:59:37.524 INFO:teuthology.orchestra.run.smithi001.stdout:5950 punch from 0x4a035ab to 0x4a0670e, (0x3163 bytes) 2024-09-13T04:59:37.524 INFO:teuthology.orchestra.run.smithi001.stdout:5952 writesame 0xcae8a58 thru 0xcaf8541 data_size 0x1(0xfaea bytes) 2024-09-13T04:59:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:5956 punch from 0x87e84de to 0x87e9bed, (0x170f bytes) 2024-09-13T04:59:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:5957 punch from 0xdafc838 to 0xdaff945, (0x310d bytes) 2024-09-13T04:59:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:5958 punch from 0x11051d2 to 0x111107b, (0xbea9 bytes) 2024-09-13T04:59:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:5959 read 0x18cfdcc thru 0x18da9a6 (0xabdb bytes) 2024-09-13T04:59:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:5960 read 0xb469a50 thru 0xb46e69d (0x4c4e bytes) 2024-09-13T04:59:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:5962 write 0x44b743c thru 0x44bccaf (0x5874 bytes) 2024-09-13T04:59:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:5964 write 0x677ec9 thru 0x67f10e (0x7246 bytes) 2024-09-13T04:59:37.592 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:37.586+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:5965 trunc from 0xeb1d959 to 0xae7515e 2024-09-13T04:59:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:5966 read 0x88294d9 thru 0x882d3c1 (0x3ee9 bytes) 2024-09-13T04:59:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:5967 trunc from 0xae7515e to 0x4cd7813 2024-09-13T04:59:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:5968 write 0x3a70f33 thru 0x3a75150 (0x421e bytes) 2024-09-13T04:59:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:37.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:5969 writesame 0xbdc793f thru 0xbdd2df2 data_size 0xc(0xb4b4 bytes) 2024-09-13T04:59:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:5971 read 0x4f91233 thru 0x4f99a7c (0x884a bytes) 2024-09-13T04:59:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:5972 writesame 0x93328e0 thru 0x9335d7f data_size 0x10(0x34a0 bytes) 2024-09-13T04:59:37.712 INFO:teuthology.orchestra.run.smithi001.stdout:5975 write 0xb30637f thru 0xb307d9d (0x1a1f bytes) 2024-09-13T04:59:37.713 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:37.706+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:37.726 INFO:teuthology.orchestra.run.smithi001.stdout:5976 punch from 0xc93983 to 0xc96daa, (0x3427 bytes) 2024-09-13T04:59:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:5977 trunc from 0xbdd2df3 to 0x305945f 2024-09-13T04:59:37.765 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.766 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.766 INFO:teuthology.orchestra.run.smithi001.stdout:5980 punch from 0x322339 to 0x328415, (0x60dc bytes) 2024-09-13T04:59:37.773 INFO:teuthology.orchestra.run.smithi001.stdout:5981 trunc from 0x305945f to 0x91f2343 2024-09-13T04:59:37.788 INFO:teuthology.orchestra.run.smithi001.stdout:5982 write 0xac1cedf thru 0xac261d7 (0x92f9 bytes) 2024-09-13T04:59:37.793 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:37.786+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:37.815 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:37.815 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:37.816 INFO:teuthology.orchestra.run.smithi001.stdout:5983 write 0xb12e448 thru 0xb13e074 (0xfc2d bytes) 2024-09-13T04:59:37.829 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:37.850 INFO:teuthology.orchestra.run.smithi001.stdout:5984 trunc from 0xb13e075 to 0x4773f76 2024-09-13T04:59:37.881 INFO:teuthology.orchestra.run.smithi001.stdout:5985 writesame 0x6bc2eec thru 0x6bd17e7 data_size 0x4a(0xe8fc bytes) 2024-09-13T04:59:37.882 INFO:teuthology.orchestra.run.smithi001.stdout:5986 punch from 0x16c534f to 0x16d3d8c, (0xea3d bytes) 2024-09-13T04:59:37.882 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.882 INFO:teuthology.orchestra.run.smithi001.stdout:5988 writesame 0x22bc960 thru 0x22c923e data_size 0x3d(0xc8df bytes) 2024-09-13T04:59:37.894 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:37.890+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:37.913 INFO:teuthology.orchestra.run.smithi001.stdout:5989 write 0x32fd5ee thru 0x33070d8 (0x9aeb bytes) 2024-09-13T04:59:37.943 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.944 INFO:teuthology.orchestra.run.smithi001.stdout:5991 read 0x1f9307c thru 0x1f96d73 (0x3cf8 bytes) 2024-09-13T04:59:37.958 INFO:teuthology.orchestra.run.smithi001.stdout:5992 writesame 0xacffe6d thru 0xad0ddf6 data_size 0x47(0xdf8a bytes) 2024-09-13T04:59:37.959 INFO:teuthology.orchestra.run.smithi001.stdout:5993 trunc from 0xad0ddf7 to 0x9f5d8de 2024-09-13T04:59:37.967 INFO:teuthology.orchestra.run.smithi001.stdout:5995 read 0x5190d8d thru 0x519f492 (0xe706 bytes) 2024-09-13T04:59:37.968 INFO:teuthology.orchestra.run.smithi001.stdout:compare and write disabled 2024-09-13T04:59:37.968 INFO:teuthology.orchestra.run.smithi001.stdout:5997 read 0x20a31e8 thru 0x20a5ea9 (0x2cc2 bytes) 2024-09-13T04:59:37.969 INFO:teuthology.orchestra.run.smithi001.stdout:5998 read 0x3cfc36e thru 0x3d0618d (0x9e20 bytes) 2024-09-13T04:59:37.974 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:37.974 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:37.979 INFO:teuthology.orchestra.run.smithi001.stdout:5999 write 0xbf5bdaf thru 0xbf5c6f2 (0x944 bytes) 2024-09-13T04:59:37.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:37.989 INFO:teuthology.orchestra.run.smithi001.stdout:6000 write 0x1c06f6 thru 0x1c31f7 (0x2b02 bytes) 2024-09-13T04:59:37.996 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:37.990+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:38.069 INFO:teuthology.orchestra.run.smithi001.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-13T04:59:38.096 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:38.090+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:38.126 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:38.126 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:38.141 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:38.229 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:38.226+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:38.298 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:38.294+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:38.320 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:38.320 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:38.335 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:38.425 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:38.422+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:38.493 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:38.493 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:38.500 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:38.494+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:38.507 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:38.602 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:38.598+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:38.638 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:38.638 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:38.652 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:38.703 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:38.698+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:38.776 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:38.777 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:38.791 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:38.833 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:38.830+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:38.905 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:38.902+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:38.954 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:38.954 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:38.968 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:39.017 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:39.014+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:39.088 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:39.088 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:39.102 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:39.145 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:39.142+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:39.208 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:39.202+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:39.243 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:39.243 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:39.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T04:59:39.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:59:39.286 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:39.286 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:39.300 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:39.345 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:39.342+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:39.410 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:39.406+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:39.473 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:39.473 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:39.487 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:39.499 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:39.546 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:39.542+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:39.575 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T04:59:39.576 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T04:59:39.596 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-13T04:59:39.220021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1180072529,"num_objects":2225,"num_object_clones":526,"num_object_copies":4450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2225,"num_whiteouts":972,"num_read":150151,"num_read_kb":637019,"num_write":18781,"num_write_kb":342774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14917,"ondisk_log_size":14917,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":383172,"kb_used_data":79064,"kb_used_omap":12,"kb_used_meta":303731,"kb_avail":754591548,"statfs":{"total":773094113280,"available":772701745152,"internally_reserved":0,"allocated":80961536,"data_stored":545891577,"data_compressed":23191092,"data_compressed_allocated":53407744,"data_compressed_original":529622058,"omap_allocated":12743,"internal_metadata":311021113},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38131601,"num_objects":218,"num_object_clones":47,"num_object_copies":436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":218,"num_whiteouts":59,"num_read":19373,"num_read_kb":32043,"num_write":2357,"num_write_kb":17801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.025030"},"pg_stats":[{"pgid":"3.1f","version":"94'438","reported_seq":2724,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:36.828115+0000","last_change":"2024-09-13T04:58:05.926696+0000","last_active":"2024-09-13T04:59:36.828115+0000","last_peered":"2024-09-13T04:59:36.828115+0000","last_clean":"2024-09-13T04:59:36.828115+0000","last_became_active":"2024-09-13T04:58:03.927975+0000","last_became_peered":"2024-09-13T04:58:03.927975+0000","last_unstale":"2024-09-13T04:59:36.828115+0000","last_undegraded":"2024-09-13T04:59:36.828115+0000","last_fullsized":"2024-09-13T04:59:36.828115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:22.475898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052405500000000005,"stat_sum":{"num_bytes":55895083,"num_objects":75,"num_object_clones":21,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":3796,"num_read_kb":15776,"num_write":491,"num_write_kb":9754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"94'293","reported_seq":1830,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:36.780441+0000","last_change":"2024-09-13T04:58:05.926269+0000","last_active":"2024-09-13T04:59:36.780441+0000","last_peered":"2024-09-13T04:59:36.780441+0000","last_clean":"2024-09-13T04:59:36.780441+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T04:59:36.780441+0000","last_undegraded":"2024-09-13T04:59:36.780441+0000","last_fullsized":"2024-09-13T04:59:36.780441+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026718299999999999,"stat_sum":{"num_bytes":23179268,"num_objects":69,"num_object_clones":14,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":25,"num_read":2424,"num_read_kb":18059,"num_write":278,"num_write_kb":8791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"95'467","reported_seq":3490,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.603936+0000","last_change":"2024-09-13T04:58:05.916753+0000","last_active":"2024-09-13T04:59:37.603936+0000","last_peered":"2024-09-13T04:59:37.603936+0000","last_clean":"2024-09-13T04:59:37.603936+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T04:59:37.603936+0000","last_undegraded":"2024-09-13T04:59:37.603936+0000","last_fullsized":"2024-09-13T04:59:37.603936+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001701801,"stat_sum":{"num_bytes":55853753,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":35,"num_read":4102,"num_read_kb":21984,"num_write":630,"num_write_kb":13734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"95'491","reported_seq":1262,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.940773+0000","last_change":"2024-09-13T04:58:05.923680+0000","last_active":"2024-09-13T04:59:37.940773+0000","last_peered":"2024-09-13T04:59:37.940773+0000","last_clean":"2024-09-13T04:59:37.940773+0000","last_became_active":"2024-09-13T04:58:03.916298+0000","last_became_peered":"2024-09-13T04:58:03.916298+0000","last_unstale":"2024-09-13T04:59:37.940773+0000","last_undegraded":"2024-09-13T04:59:37.940773+0000","last_fullsized":"2024-09-13T04:59:37.940773+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:15.064934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070208100000000002,"stat_sum":{"num_bytes":34967678,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":1847,"num_read_kb":7952,"num_write":621,"num_write_kb":7913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"95'437","reported_seq":4800,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.928221+0000","last_change":"2024-09-13T04:58:05.912761+0000","last_active":"2024-09-13T04:59:37.928221+0000","last_peered":"2024-09-13T04:59:37.928221+0000","last_clean":"2024-09-13T04:59:37.928221+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T04:59:37.928221+0000","last_undegraded":"2024-09-13T04:59:37.928221+0000","last_fullsized":"2024-09-13T04:59:37.928221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047640200000000001,"stat_sum":{"num_bytes":31860432,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":25,"num_read":5548,"num_read_kb":17529,"num_write":506,"num_write_kb":6755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"95'405","reported_seq":3047,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.881018+0000","last_change":"2024-09-13T04:58:05.924255+0000","last_active":"2024-09-13T04:59:37.881018+0000","last_peered":"2024-09-13T04:59:37.881018+0000","last_clean":"2024-09-13T04:59:37.881018+0000","last_became_active":"2024-09-13T04:58:03.930844+0000","last_became_peered":"2024-09-13T04:58:03.930844+0000","last_unstale":"2024-09-13T04:59:37.881018+0000","last_undegraded":"2024-09-13T04:59:37.881018+0000","last_fullsized":"2024-09-13T04:59:37.881018+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:22:26.309845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011353610000000001,"stat_sum":{"num_bytes":36387449,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":28,"num_read":4046,"num_read_kb":13158,"num_write":481,"num_write_kb":11695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"95'395","reported_seq":18458,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.856339+0000","last_change":"2024-09-13T04:58:05.912545+0000","last_active":"2024-09-13T04:59:37.856339+0000","last_peered":"2024-09-13T04:59:37.856339+0000","last_clean":"2024-09-13T04:59:37.856339+0000","last_became_active":"2024-09-13T04:58:03.917085+0000","last_became_peered":"2024-09-13T04:58:03.917085+0000","last_unstale":"2024-09-13T04:59:37.856339+0000","last_undegraded":"2024-09-13T04:59:37.856339+0000","last_fullsized":"2024-09-13T04:59:37.856339+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:45:01.603707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032380400000000002,"stat_sum":{"num_bytes":48245943,"num_objects":80,"num_object_clones":20,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":35,"num_read":19047,"num_read_kb":95950,"num_write":464,"num_write_kb":19269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"94'390","reported_seq":2747,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:36.914807+0000","last_change":"2024-09-13T04:58:05.926985+0000","last_active":"2024-09-13T04:59:36.914807+0000","last_peered":"2024-09-13T04:59:36.914807+0000","last_clean":"2024-09-13T04:59:36.914807+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T04:59:36.914807+0000","last_undegraded":"2024-09-13T04:59:36.914807+0000","last_fullsized":"2024-09-13T04:59:36.914807+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129199999999999,"stat_sum":{"num_bytes":28075154,"num_objects":70,"num_object_clones":21,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":33,"num_read":3204,"num_read_kb":9798,"num_write":500,"num_write_kb":5964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"94'255","reported_seq":3475,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:36.892475+0000","last_change":"2024-09-13T04:58:05.926437+0000","last_active":"2024-09-13T04:59:36.892475+0000","last_peered":"2024-09-13T04:59:36.892475+0000","last_clean":"2024-09-13T04:59:36.892475+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T04:59:36.892475+0000","last_undegraded":"2024-09-13T04:59:36.892475+0000","last_fullsized":"2024-09-13T04:59:36.892475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035256799999999999,"stat_sum":{"num_bytes":33822718,"num_objects":70,"num_object_clones":18,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":3213,"num_read_kb":21668,"num_write":266,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"95'477","reported_seq":2199,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.861585+0000","last_change":"2024-09-13T04:58:05.919167+0000","last_active":"2024-09-13T04:59:37.861585+0000","last_peered":"2024-09-13T04:59:37.861585+0000","last_clean":"2024-09-13T04:59:37.861585+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T04:59:37.861585+0000","last_undegraded":"2024-09-13T04:59:37.861585+0000","last_fullsized":"2024-09-13T04:59:37.861585+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00167639,"stat_sum":{"num_bytes":31344092,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":3160,"num_read_kb":19548,"num_write":616,"num_write_kb":7267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"95'346","reported_seq":896,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.856049+0000","last_change":"2024-09-13T04:58:05.929029+0000","last_active":"2024-09-13T04:59:37.856049+0000","last_peered":"2024-09-13T04:59:37.856049+0000","last_clean":"2024-09-13T04:59:37.856049+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T04:59:37.856049+0000","last_undegraded":"2024-09-13T04:59:37.856049+0000","last_fullsized":"2024-09-13T04:59:37.856049+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038806099999999998,"stat_sum":{"num_bytes":22342863,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":26,"num_read":1356,"num_read_kb":10511,"num_write":415,"num_write_kb":6053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"95'639","reported_seq":4009,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.746248+0000","last_change":"2024-09-13T04:58:05.917113+0000","last_active":"2024-09-13T04:59:37.746248+0000","last_peered":"2024-09-13T04:59:37.746248+0000","last_clean":"2024-09-13T04:59:37.746248+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T04:59:37.746248+0000","last_undegraded":"2024-09-13T04:59:37.746248+0000","last_fullsized":"2024-09-13T04:59:37.746248+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019474449999999999,"stat_sum":{"num_bytes":22830739,"num_objects":76,"num_object_clones":19,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":34,"num_read":5387,"num_read_kb":13494,"num_write":882,"num_write_kb":5061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:35.348330+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T04:59:35.348330+0000","last_peered":"2024-09-13T04:59:35.348330+0000","last_clean":"2024-09-13T04:59:35.348330+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T04:59:35.348330+0000","last_undegraded":"2024-09-13T04:59:35.348330+0000","last_fullsized":"2024-09-13T04:59:35.348330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"95'251","reported_seq":2371,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.756635+0000","last_change":"2024-09-13T04:58:19.436873+0000","last_active":"2024-09-13T04:59:37.756635+0000","last_peered":"2024-09-13T04:59:37.756635+0000","last_clean":"2024-09-13T04:59:37.756635+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T04:59:37.756635+0000","last_undegraded":"2024-09-13T04:59:37.756635+0000","last_fullsized":"2024-09-13T04:59:37.756635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058494600000000003,"stat_sum":{"num_bytes":51022214,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":31,"num_read":2150,"num_read_kb":23091,"num_write":268,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:35.214597+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T04:59:35.214597+0000","last_peered":"2024-09-13T04:59:35.214597+0000","last_clean":"2024-09-13T04:59:35.214597+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T04:59:35.214597+0000","last_undegraded":"2024-09-13T04:59:35.214597+0000","last_fullsized":"2024-09-13T04:59:35.214597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"95'650","reported_seq":1816,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.857647+0000","last_change":"2024-09-13T04:58:05.913131+0000","last_active":"2024-09-13T04:59:37.857647+0000","last_peered":"2024-09-13T04:59:37.857647+0000","last_clean":"2024-09-13T04:59:37.857647+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T04:59:37.857647+0000","last_undegraded":"2024-09-13T04:59:37.857647+0000","last_fullsized":"2024-09-13T04:59:37.857647+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072848599999999998,"stat_sum":{"num_bytes":26385581,"num_objects":57,"num_object_clones":9,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":4142,"num_read_kb":12334,"num_write":923,"num_write_kb":8377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:35.348233+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T04:59:35.348233+0000","last_peered":"2024-09-13T04:59:35.348233+0000","last_clean":"2024-09-13T04:59:35.348233+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T04:59:35.348233+0000","last_undegraded":"2024-09-13T04:59:35.348233+0000","last_fullsized":"2024-09-13T04:59:35.348233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"94'519","reported_seq":1330,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:36.844930+0000","last_change":"2024-09-13T04:59:29.180359+0000","last_active":"2024-09-13T04:59:36.844930+0000","last_peered":"2024-09-13T04:59:36.844930+0000","last_clean":"2024-09-13T04:59:36.844930+0000","last_became_active":"2024-09-13T04:59:29.177938+0000","last_became_peered":"2024-09-13T04:59:29.177938+0000","last_unstale":"2024-09-13T04:59:36.844930+0000","last_undegraded":"2024-09-13T04:59:36.844930+0000","last_fullsized":"2024-09-13T04:59:36.844930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:09.883322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069520900000000002,"stat_sum":{"num_bytes":25705523,"num_objects":76,"num_object_clones":13,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":37,"num_read":2720,"num_read_kb":14627,"num_write":717,"num_write_kb":8954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":87,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T04:59:34.199622+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T04:59:34.199622+0000","last_peered":"2024-09-13T04:59:34.199622+0000","last_clean":"2024-09-13T04:59:34.199622+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T04:59:34.199622+0000","last_undegraded":"2024-09-13T04:59:34.199622+0000","last_fullsized":"2024-09-13T04:59:34.199622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"95'458","reported_seq":1954,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.757803+0000","last_change":"2024-09-13T04:58:05.919061+0000","last_active":"2024-09-13T04:59:37.757803+0000","last_peered":"2024-09-13T04:59:37.757803+0000","last_clean":"2024-09-13T04:59:37.757803+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T04:59:37.757803+0000","last_undegraded":"2024-09-13T04:59:37.757803+0000","last_fullsized":"2024-09-13T04:59:37.757803+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044299999999998,"stat_sum":{"num_bytes":19182227,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":29,"num_read":3392,"num_read_kb":10449,"num_write":522,"num_write_kb":6320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"95'591","reported_seq":3904,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.640154+0000","last_change":"2024-09-13T04:58:05.928769+0000","last_active":"2024-09-13T04:59:37.640154+0000","last_peered":"2024-09-13T04:59:37.640154+0000","last_clean":"2024-09-13T04:59:37.640154+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T04:59:37.640154+0000","last_undegraded":"2024-09-13T04:59:37.640154+0000","last_fullsized":"2024-09-13T04:59:37.640154+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059983499999999995,"stat_sum":{"num_bytes":52581994,"num_objects":86,"num_object_clones":22,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":36,"num_read":6087,"num_read_kb":21794,"num_write":708,"num_write_kb":15545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":90,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:35.348198+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T04:59:35.348198+0000","last_peered":"2024-09-13T04:59:35.348198+0000","last_clean":"2024-09-13T04:59:35.348198+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T04:59:35.348198+0000","last_undegraded":"2024-09-13T04:59:35.348198+0000","last_fullsized":"2024-09-13T04:59:35.348198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"94'774","reported_seq":1842,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:36.940083+0000","last_change":"2024-09-13T04:58:05.926694+0000","last_active":"2024-09-13T04:59:36.940083+0000","last_peered":"2024-09-13T04:59:36.940083+0000","last_clean":"2024-09-13T04:59:36.940083+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T04:59:36.940083+0000","last_undegraded":"2024-09-13T04:59:36.940083+0000","last_fullsized":"2024-09-13T04:59:36.940083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035933999999999998,"stat_sum":{"num_bytes":38958699,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":33,"num_read":4560,"num_read_kb":18755,"num_write":1127,"num_write_kb":10947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:35.364321+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T04:59:35.364321+0000","last_peered":"2024-09-13T04:59:35.364321+0000","last_clean":"2024-09-13T04:59:35.364321+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T04:59:35.364321+0000","last_undegraded":"2024-09-13T04:59:35.364321+0000","last_fullsized":"2024-09-13T04:59:35.364321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"95'356","reported_seq":4675,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.912091+0000","last_change":"2024-09-13T04:58:05.976100+0000","last_active":"2024-09-13T04:59:37.912091+0000","last_peered":"2024-09-13T04:59:37.912091+0000","last_clean":"2024-09-13T04:59:37.912091+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T04:59:37.912091+0000","last_undegraded":"2024-09-13T04:59:37.912091+0000","last_fullsized":"2024-09-13T04:59:37.912091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055057000000000001,"stat_sum":{"num_bytes":46515576,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":28,"num_read":5347,"num_read_kb":22870,"num_write":373,"num_write_kb":9846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:35.347655+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T04:59:35.347655+0000","last_peered":"2024-09-13T04:59:35.347655+0000","last_clean":"2024-09-13T04:59:35.347655+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T04:59:35.347655+0000","last_undegraded":"2024-09-13T04:59:35.347655+0000","last_fullsized":"2024-09-13T04:59:35.347655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"95'896","reported_seq":2724,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.804561+0000","last_change":"2024-09-13T04:58:05.912947+0000","last_active":"2024-09-13T04:59:37.804561+0000","last_peered":"2024-09-13T04:59:37.804561+0000","last_clean":"2024-09-13T04:59:37.804561+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T04:59:37.804561+0000","last_undegraded":"2024-09-13T04:59:37.804561+0000","last_fullsized":"2024-09-13T04:59:37.804561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063981800000000001,"stat_sum":{"num_bytes":32501571,"num_objects":64,"num_object_clones":8,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":31,"num_read":6226,"num_read_kb":24426,"num_write":1328,"num_write_kb":11459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":89,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:35.347852+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T04:59:35.347852+0000","last_peered":"2024-09-13T04:59:35.347852+0000","last_clean":"2024-09-13T04:59:35.347852+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T04:59:35.347852+0000","last_undegraded":"2024-09-13T04:59:35.347852+0000","last_fullsized":"2024-09-13T04:59:35.347852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":235,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T04:59:35.214586+0000","last_change":"2024-09-13T04:57:50.134253+0000","last_active":"2024-09-13T04:59:35.214586+0000","last_peered":"2024-09-13T04:59:35.214586+0000","last_clean":"2024-09-13T04:59:35.214586+0000","last_became_active":"2024-09-13T04:57:50.132771+0000","last_became_peered":"2024-09-13T04:57:50.132771+0000","last_unstale":"2024-09-13T04:59:35.214586+0000","last_undegraded":"2024-09-13T04:59:35.214586+0000","last_fullsized":"2024-09-13T04:59:35.214586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:06:01.350459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"95'395","reported_seq":3373,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.725830+0000","last_change":"2024-09-13T04:58:05.912570+0000","last_active":"2024-09-13T04:59:37.725830+0000","last_peered":"2024-09-13T04:59:37.725830+0000","last_clean":"2024-09-13T04:59:37.725830+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T04:59:37.725830+0000","last_undegraded":"2024-09-13T04:59:37.725830+0000","last_fullsized":"2024-09-13T04:59:37.725830+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000378859,"stat_sum":{"num_bytes":53298612,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":31,"num_read":4349,"num_read_kb":11171,"num_write":456,"num_write_kb":10359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"95'524","reported_seq":5689,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.641928+0000","last_change":"2024-09-13T04:58:05.917713+0000","last_active":"2024-09-13T04:59:37.641928+0000","last_peered":"2024-09-13T04:59:37.641928+0000","last_clean":"2024-09-13T04:59:37.641928+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T04:59:37.641928+0000","last_undegraded":"2024-09-13T04:59:37.641928+0000","last_fullsized":"2024-09-13T04:59:37.641928+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023808219999999999,"stat_sum":{"num_bytes":29679324,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":29,"num_read":6708,"num_read_kb":22288,"num_write":663,"num_write_kb":12014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"95'387","reported_seq":6471,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.809271+0000","last_change":"2024-09-13T04:58:05.918413+0000","last_active":"2024-09-13T04:59:37.809271+0000","last_peered":"2024-09-13T04:59:37.809271+0000","last_clean":"2024-09-13T04:59:37.809271+0000","last_became_active":"2024-09-13T04:58:03.917664+0000","last_became_peered":"2024-09-13T04:58:03.917664+0000","last_unstale":"2024-09-13T04:59:37.809271+0000","last_undegraded":"2024-09-13T04:59:37.809271+0000","last_fullsized":"2024-09-13T04:59:37.809271+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:58.330434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027083089999999999,"stat_sum":{"num_bytes":37917692,"num_objects":79,"num_object_clones":24,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":32,"num_read":6329,"num_read_kb":24105,"num_write":415,"num_write_kb":14042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"95'270","reported_seq":1888,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.928081+0000","last_change":"2024-09-13T04:58:05.934266+0000","last_active":"2024-09-13T04:59:37.928081+0000","last_peered":"2024-09-13T04:59:37.928081+0000","last_clean":"2024-09-13T04:59:37.928081+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T04:59:37.928081+0000","last_undegraded":"2024-09-13T04:59:37.928081+0000","last_fullsized":"2024-09-13T04:59:37.928081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092250299999999997,"stat_sum":{"num_bytes":34794320,"num_objects":72,"num_object_clones":17,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":29,"num_read":1650,"num_read_kb":17074,"num_write":279,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"95'596","reported_seq":1504,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.907724+0000","last_change":"2024-09-13T04:58:05.928679+0000","last_active":"2024-09-13T04:59:37.907724+0000","last_peered":"2024-09-13T04:59:37.907724+0000","last_clean":"2024-09-13T04:59:37.907724+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T04:59:37.907724+0000","last_undegraded":"2024-09-13T04:59:37.907724+0000","last_fullsized":"2024-09-13T04:59:37.907724+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000503628,"stat_sum":{"num_bytes":55011671,"num_objects":75,"num_object_clones":20,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":36,"num_read":2828,"num_read_kb":9072,"num_write":819,"num_write_kb":7825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"95'381","reported_seq":4128,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.828431+0000","last_change":"2024-09-13T04:58:05.917743+0000","last_active":"2024-09-13T04:59:37.828431+0000","last_peered":"2024-09-13T04:59:37.828431+0000","last_clean":"2024-09-13T04:59:37.828431+0000","last_became_active":"2024-09-13T04:58:03.918196+0000","last_became_peered":"2024-09-13T04:58:03.918196+0000","last_unstale":"2024-09-13T04:59:37.828431+0000","last_undegraded":"2024-09-13T04:59:37.828431+0000","last_fullsized":"2024-09-13T04:59:37.828431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:53:56.669192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340071,"stat_sum":{"num_bytes":41049950,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":19,"num_read":5427,"num_read_kb":35374,"num_write":569,"num_write_kb":15007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"95'354","reported_seq":1512,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.880701+0000","last_change":"2024-09-13T04:58:05.919748+0000","last_active":"2024-09-13T04:59:37.880701+0000","last_peered":"2024-09-13T04:59:37.880701+0000","last_clean":"2024-09-13T04:59:37.880701+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T04:59:37.880701+0000","last_undegraded":"2024-09-13T04:59:37.880701+0000","last_fullsized":"2024-09-13T04:59:37.880701+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001918548,"stat_sum":{"num_bytes":22954929,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":24,"num_read":2416,"num_read_kb":8688,"num_write":350,"num_write_kb":5088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"95'750","reported_seq":7899,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.861550+0000","last_change":"2024-09-13T04:58:05.928992+0000","last_active":"2024-09-13T04:59:37.861550+0000","last_peered":"2024-09-13T04:59:37.861550+0000","last_clean":"2024-09-13T04:59:37.861550+0000","last_became_active":"2024-09-13T04:58:03.917283+0000","last_became_peered":"2024-09-13T04:58:03.917283+0000","last_unstale":"2024-09-13T04:59:37.861550+0000","last_undegraded":"2024-09-13T04:59:37.861550+0000","last_fullsized":"2024-09-13T04:59:37.861550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:26.197189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057226800000000002,"stat_sum":{"num_bytes":34896906,"num_objects":85,"num_object_clones":24,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":38,"num_read":9654,"num_read_kb":25615,"num_write":1090,"num_write_kb":19474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"95'357","reported_seq":6446,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.861187+0000","last_change":"2024-09-13T04:58:05.923716+0000","last_active":"2024-09-13T04:59:37.861187+0000","last_peered":"2024-09-13T04:59:37.861187+0000","last_clean":"2024-09-13T04:59:37.861187+0000","last_became_active":"2024-09-13T04:58:03.935847+0000","last_became_peered":"2024-09-13T04:58:03.935847+0000","last_unstale":"2024-09-13T04:59:37.861187+0000","last_undegraded":"2024-09-13T04:59:37.861187+0000","last_fullsized":"2024-09-13T04:59:37.861187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:18.318944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063129600000000005,"stat_sum":{"num_bytes":46289868,"num_objects":91,"num_object_clones":23,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":43,"num_read":6883,"num_read_kb":14642,"num_write":383,"num_write_kb":7396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"95'343","reported_seq":4796,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.945182+0000","last_change":"2024-09-13T04:58:05.916560+0000","last_active":"2024-09-13T04:59:37.945182+0000","last_peered":"2024-09-13T04:59:37.945182+0000","last_clean":"2024-09-13T04:59:37.945182+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T04:59:37.945182+0000","last_undegraded":"2024-09-13T04:59:37.945182+0000","last_fullsized":"2024-09-13T04:59:37.945182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016504969999999999,"stat_sum":{"num_bytes":58239164,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":27,"num_read":5197,"num_read_kb":31241,"num_write":362,"num_write_kb":16398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"95'369","reported_seq":3318,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.925078+0000","last_change":"2024-09-13T04:58:05.976061+0000","last_active":"2024-09-13T04:59:37.925078+0000","last_peered":"2024-09-13T04:59:37.925078+0000","last_clean":"2024-09-13T04:59:37.925078+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T04:59:37.925078+0000","last_undegraded":"2024-09-13T04:59:37.925078+0000","last_fullsized":"2024-09-13T04:59:37.925078+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055531999999999999,"stat_sum":{"num_bytes":24935871,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":29,"num_read":3771,"num_read_kb":13103,"num_write":386,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"95'543","reported_seq":1641,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-13T04:59:37.858128+0000","last_change":"2024-09-13T04:58:05.918244+0000","last_active":"2024-09-13T04:59:37.858128+0000","last_peered":"2024-09-13T04:59:37.858128+0000","last_clean":"2024-09-13T04:59:37.858128+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T04:59:37.858128+0000","last_undegraded":"2024-09-13T04:59:37.858128+0000","last_fullsized":"2024-09-13T04:59:37.858128+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029379390000000001,"stat_sum":{"num_bytes":22755278,"num_objects":67,"num_object_clones":18,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":3069,"num_read_kb":10773,"num_write":732,"num_write_kb":7399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1179482142,"num_objects":2221,"num_object_clones":526,"num_object_copies":4442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2221,"num_whiteouts":972,"num_read":150035,"num_read_kb":636919,"num_write":18620,"num_write_kb":340356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73134080,"data_stored":543578626,"data_compressed":23287544,"data_compressed_allocated":53743616,"data_compressed_original":530838570,"omap_allocated":0,"internal_metadata":0},"log_size":14797,"ondisk_log_size":14797,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12454,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40760,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331080,"statfs":{"total":96636764160,"available":96595025920,"internally_reserved":0,"allocated":5685248,"data_stored":33596840,"data_compressed":1541896,"data_compressed_allocated":3321856,"data_compressed_original":32288491,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.109},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.0029999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":1.294},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.0780000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.91800000000000004}]}]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48708,"kb_used_data":9400,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94323132,"statfs":{"total":96636764160,"available":96586887168,"internally_reserved":0,"allocated":9625600,"data_stored":68614994,"data_compressed":2749646,"data_compressed_allocated":6209536,"data_compressed_original":66622248,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.96399999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.88800000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.3520000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.026}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.508},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.3089999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509466,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52928,"kb_used_data":9452,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94318912,"statfs":{"total":96636764160,"available":96582565888,"internally_reserved":0,"allocated":9678848,"data_stored":66737857,"data_compressed":2670341,"data_compressed_allocated":6651904,"data_compressed_original":65021801,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.501}]},{"osd":2,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":11.266,"5min":11.266,"15min":11.266},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.35299999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.73699999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:58:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.69999999999999996}]}]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43372,"kb_used_data":12260,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94328468,"statfs":{"total":96636764160,"available":96592351232,"internally_reserved":0,"allocated":12554240,"data_stored":81977872,"data_compressed":3569565,"data_compressed_allocated":8409088,"data_compressed_original":79452173,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.29099999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.34599999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.46000000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.45500000000000002}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.73699999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.94299999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40868,"kb_used_data":9708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94330972,"statfs":{"total":96636764160,"available":96594915328,"internally_reserved":0,"allocated":9940992,"data_stored":67003205,"data_compressed":2855401,"data_compressed_allocated":6402048,"data_compressed_original":64954973,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.31},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.443}]},{"osd":1,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.5},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.621},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Sep 13 04:58:50 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":0.54300000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43676,"kb_used_data":12564,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94328164,"statfs":{"total":96636764160,"available":96592039936,"internally_reserved":0,"allocated":12865536,"data_stored":89092718,"data_compressed":3178192,"data_compressed_allocated":8978432,"data_compressed_original":86745652,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.85599999999999998}]},{"osd":5,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:58:51 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.748},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.91600000000000004}]}]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59940,"kb_used_data":10608,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":10862592,"data_stored":70543511,"data_compressed":2982146,"data_compressed_allocated":6959104,"data_compressed_original":68180252,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.377},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":15.755000000000001,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.5},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":15.802,"5min":15.802,"15min":15.802},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1,"5min":1,"15min":1},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":1.794,"15min":1.794},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":15.859999999999999,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":11.474,"5min":11.474,"15min":11.474},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":11.528,"5min":11.528,"15min":11.528},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.897,"5min":1.897,"15min":1.897},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":15.885999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.876}]},{"osd":7,"last update":"Fri Sep 13 04:58:47 2024","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":15.834,"5min":15.834,"15min":15.834},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.66000000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52920,"kb_used_data":9520,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94318920,"statfs":{"total":96636764160,"available":96582574080,"internally_reserved":0,"allocated":9748480,"data_stored":68324580,"data_compressed":3643905,"data_compressed_allocated":6475776,"data_compressed_original":66356468,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.5250000000000004,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":4.633,"5min":4.633,"15min":4.633},"last":0.99099999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.0960000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.6989999999999998,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.7599999999999998,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:58:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.875}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9777152,"data_stored":70060121,"data_compressed":2982146,"data_compressed_allocated":6959104,"data_compressed_original":68180252,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11702272,"data_stored":88001829,"data_compressed":3171965,"data_compressed_allocated":8941568,"data_compressed_original":86172212,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8908800,"data_stored":66692915,"data_compressed":2863504,"data_compressed_allocated":6430720,"data_compressed_original":65114717,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11886592,"data_stored":82651157,"data_compressed":3602709,"data_compressed_allocated":8585216,"data_compressed_original":80422925,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8519680,"data_stored":66226812,"data_compressed":2670341,"data_compressed_allocated":6651904,"data_compressed_original":65021801,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8806400,"data_stored":67732770,"data_compressed":3675691,"data_compressed_allocated":6496256,"data_compressed_original":66209012,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8679424,"data_stored":68518083,"data_compressed":2763342,"data_compressed_allocated":6270976,"data_compressed_original":66949928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4853760,"data_stored":33694939,"data_compressed":1557846,"data_compressed_allocated":3407872,"data_compressed_original":32767723,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T04:59:39.611 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:39.606+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:39.648 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:39.648 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:39.651 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:39.651 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:39.662 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:39.668 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:39.668 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:39.685 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:39.685 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:39.701 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:39.701 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:39.720 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:39.721 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:39.737 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:39.738 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:39.749 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:39.746+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:39.757 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:39.757 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:39.775 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:39.776 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:39.813 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:39.810+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:39.828 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:39.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:39.829 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:39.829 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:39.829 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:39.843 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:39.914 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:39.910+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:40.013 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:40.013 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:40.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:40.013 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:40.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211802544999998, 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:40.018 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211768290999998, 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:40.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.022 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211737389999996, 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.023 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.027 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211685509000006, 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.028 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:40.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211653706000007, 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.032 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:40.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:40.036 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211619752999994, 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.037 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:40.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211572016999995, 2024-09-13T04:59:40.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211532454999997, 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211484760999994, 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211421677999994, 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211385243999999, 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.059 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.063 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211351320999995, 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:40.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:40.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211283108000003, 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.068 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:40.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.072 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211249402000007, 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:40.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.211199210000004, 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.077 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.210893177000003, 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.081 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.085 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.210843975000003, 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:40.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:40.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.210795860999994, 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.090 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:40.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.210744383999995, 2024-09-13T04:59:40.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:40.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:40.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.098 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 76.210709303000002, 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:40.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:40.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.103 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:40.105 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:40.105 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:40.102+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:40.205 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:40.202+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:59:40.285 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:40.285 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:40.285 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:40.285 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:40.300 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:40.318 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:40.314+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:40.408 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:40.402+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:40.471 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:40.471 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:40.471 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:40.471 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:40.471 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:40.485 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:40.510 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:40.506+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:40.610 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:40.610 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99147 3.8 e92/24 3:17747dba:::rbd_data.112afaf35b85.0000000000000004:head v 92'362, mlcod=92'360)", 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2248614230000001, 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092626132, 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.611 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382240+0000", 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.612 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384691+0000", 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024508730000000001 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384738+0000", 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7345999999999997e-05 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384760+0000", 2024-09-13T04:59:40.613 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1330000000000001e-05 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384885+0000", 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012574299999999999 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384941+0000", 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5210000000000002e-05 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:40.614 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474807+0000", 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089866434999999995 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474870+0000", 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2692000000000002e-05 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99148 3.8 e92/24 3:17747dba:::rbd_data.112afaf35b85.0000000000000004:head v 92'364, mlcod=92'360)", 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:40.615 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.2014997249999997, 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.074491979999999999, 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:40.616 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405603+0000", 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405621+0000", 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7927e-05 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.617 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405657+0000", 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5463999999999998e-05 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405675+0000", 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7935e-05 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.408148+0000", 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024729909999999999 2024-09-13T04:59:40.618 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.408388+0000", 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240292 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480055+0000", 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.071666647 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.619 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480097+0000", 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2617000000000002e-05 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99156 3.0 e92/24 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head v 92'539, mlcod=91'537)", 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.1976728140000006, 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.070695703999999998, 2024-09-13T04:59:40.620 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.621 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409430+0000", 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412318+0000", 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028881380000000002 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412366+0000", 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7565000000000003e-05 2024-09-13T04:59:40.622 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412389+0000", 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3345000000000001e-05 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412654+0000", 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000264942 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412710+0000", 2024-09-13T04:59:40.623 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6044000000000001e-05 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480102+0000", 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.067391567999999999 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480128+0000", 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6251e-05 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.624 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99149 3.8 e92/24 3:14f8ab77:::rbd_data.112afaf35b85.0000000000000002:head v 92'365, mlcod=92'360)", 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.1971845959999996, 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.070251837999999997, 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:40.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409919+0000", 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414671+0000", 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0047520030000000003 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414735+0000", 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4139000000000004e-05 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414757+0000", 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2327000000000002e-05 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.420020+0000", 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0052631450000000003 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.420067+0000", 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6789999999999998e-05 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480133+0000", 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060066348999999998 2024-09-13T04:59:40.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480172+0000", 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8757000000000002e-05 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108618 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 30720~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2286881550000004, 2024-09-13T04:59:40.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05920533, 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108618 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378418+0000", 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.404e-06 2024-09-13T04:59:40.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378422+0000", 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6040000000000001e-06 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378425+0000", 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437429+0000", 2024-09-13T04:59:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059004369000000001 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437529+0000", 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437622+0000", 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2734999999999999e-05 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108619 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 44032~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2286762769999999, 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059431339, 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108619 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.634 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378428+0000", 2024-09-13T04:59:40.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378430+0000", 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2110000000000001e-06 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378448+0000", 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7785999999999999e-05 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378462+0000", 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4098e-05 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437678+0000", 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059216116999999999 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437764+0000", 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5902999999999994e-05 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437860+0000", 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5637999999999995e-05 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108620 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 57344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2286374760000003, 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059594146000000001, 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.638 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108620 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378473+0000", 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1480000000000003e-06 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378478+0000", 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0649999999999998e-06 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378482+0000", 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2270000000000002e-06 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437899+0000", 2024-09-13T04:59:40.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059417046000000001 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437977+0000", 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8306999999999994e-05 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438062+0000", 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4769000000000001e-05 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108621 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 72704~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2286208380000003, 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059812428000000001, 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108621 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.644 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378485+0000", 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4839999999999999e-06 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378489+0000", 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7050000000000001e-06 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378493+0000", 2024-09-13T04:59:40.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8970000000000001e-06 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438104+0000", 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059610677000000001 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438183+0000", 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438297+0000", 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000113187 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108622 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 86016~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2286075280000004, 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059970109000000001, 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.647 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108622 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378499+0000", 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3080000000000002e-06 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378505+0000", 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0920000000000003e-06 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378514+0000", 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7320000000000004e-06 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438329+0000", 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059814683 2024-09-13T04:59:40.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438396+0000", 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7063999999999994e-05 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438468+0000", 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1712999999999996e-05 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.651 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108623 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 99328~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2285877750000003, 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060190821999999998, 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108623 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.652 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:40.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378519+0000", 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.75e-06 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378523+0000", 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-13T04:59:40.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378525+0000", 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8509999999999999e-06 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438511+0000", 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059985986999999998 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438599+0000", 2024-09-13T04:59:40.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7189999999999997e-05 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438708+0000", 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109392 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108632 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 209920~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:40.656 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2259265949999998, 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059361616999999998, 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108632 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:40.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381191+0000", 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3181e-05 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381196+0000", 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7929999999999997e-06 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381199+0000", 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1820000000000002e-06 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440405+0000", 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059205525000000002 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440470+0000", 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5569999999999997e-05 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440540+0000", 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9623999999999996e-05 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108633 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 223232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2259025079999999, 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059563148000000003, 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.661 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108633 2024-09-13T04:59:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381204+0000", 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4899999999999999e-06 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381207+0000", 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5949999999999999e-06 2024-09-13T04:59:40.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381210+0000", 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9799999999999998e-06 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440582+0000", 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059371454999999997 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440666+0000", 2024-09-13T04:59:40.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4803999999999996e-05 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440766+0000", 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9130999999999997e-05 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108634 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 236544~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2258920440000001, 2024-09-13T04:59:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059731899999999997, 2024-09-13T04:59:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108634 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381214+0000", 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.327e-06 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381218+0000", 2024-09-13T04:59:40.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6100000000000002e-06 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381220+0000", 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.768e-06 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440809+0000", 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059588226000000001 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440872+0000", 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3910000000000003e-05 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440945+0000", 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2316000000000004e-05 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108635 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 249856~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.671 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2258818429999998, 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059973802, 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108635 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:40.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381224+0000", 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.406e-06 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381228+0000", 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6069999999999999e-06 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381231+0000", 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440986+0000", 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059755437000000002 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441082+0000", 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6243999999999995e-05 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441197+0000", 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114601 2024-09-13T04:59:40.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108636 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 265216~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2257841469999997, 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060179980000000001, 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.676 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108636 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381234+0000", 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999132 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381323+0000", 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9089e-05 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381331+0000", 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.401e-06 2024-09-13T04:59:40.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381334+0000", 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9369999999999997e-06 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441242+0000", 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059907795999999999 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441338+0000", 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6135e-05 2024-09-13T04:59:40.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441501+0000", 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016236899999999999 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108651 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 471040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2218254560000004, 2024-09-13T04:59:40.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059311423000000002, 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108651 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385279+0000", 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385281+0000", 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5039999999999999e-06 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385284+0000", 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7230000000000002e-06 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385287+0000", 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.084e-06 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444435+0000", 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059147995000000002 2024-09-13T04:59:40.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444514+0000", 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8023999999999997e-05 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444591+0000", 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7416000000000006e-05 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108652 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 484352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2217877919999998, 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059491817000000002, 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108652 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385319+0000", 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.804e-06 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385323+0000", 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3329999999999998e-06 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385326+0000", 2024-09-13T04:59:40.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3529999999999999e-06 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444645+0000", 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059319233999999998 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444732+0000", 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6557000000000005e-05 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444809+0000", 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6973999999999999e-05 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108669 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 194560~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2177743049999998, 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059219301000000002, 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.689 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108669 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389330+0000", 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389335+0000", 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9930000000000003e-06 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389339+0000", 2024-09-13T04:59:40.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.07e-06 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389342+0000", 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.897e-06 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448324+0000", 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.058981947999999999 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448430+0000", 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105351 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448550+0000", 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120301 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108670 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 196608~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:40.693 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.217759901, 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059399572999999997, 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108670 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389346+0000", 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.466e-06 2024-09-13T04:59:40.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389350+0000", 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6950000000000002e-06 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389353+0000", 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9330000000000001e-06 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448582+0000", 2024-09-13T04:59:40.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059229054000000003 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448634+0000", 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2299000000000002e-05 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448745+0000", 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110386 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108671 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 209920~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4.2177484960000005, 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05961288, 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108671 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:40.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:40.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389358+0000", 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.331e-06 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389361+0000", 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6820000000000001e-06 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:40.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389364+0000", 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.994e-06 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448786+0000", 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059421307999999999 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448874+0000", 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8337000000000005e-05 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:40.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448969+0000", 2024-09-13T04:59:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5486e-05 2024-09-13T04:59:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:40.702 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:40.702 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:40.702 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:40.606+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:40.703 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:40.745 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:40.742+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:40.812 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:40.806+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:40.838 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:40.839 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:40.853 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:40.913 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:40.910+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:41.014 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:41.010+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:41.024 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:41.024 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:41.024 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:41.025 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:41.025 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:41.039 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:41.130 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:41.126+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:41.212 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:41.212 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:41.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:41.212 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:41.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.212 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.212 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:41.212 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489928907999996, 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053636234999999997, 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:41.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772488+0000", 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053447254999999999 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772562+0000", 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4276999999999994e-05 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772656+0000", 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3993999999999995e-05 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5898240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489903515999998, 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054161726, 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:59:41.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772681+0000", 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053622832000000002 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772751+0000", 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0166999999999994e-05 2024-09-13T04:59:41.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773207+0000", 2024-09-13T04:59:41.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045547300000000002 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5931008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489886064999993, 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054348847999999998, 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.222 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773235+0000", 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054159997000000001 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773316+0000", 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0727000000000001e-05 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773411+0000", 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5266999999999994e-05 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.226 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489868817000001, 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054777692000000003, 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:59:41.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773440+0000", 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054341649999999998 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773514+0000", 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4501999999999999e-05 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773858+0000", 2024-09-13T04:59:41.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034316700000000002 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489846064000005, 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054958912999999998, 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773886+0000", 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054770389000000003 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773964+0000", 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774062+0000", 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7059999999999996e-05 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.235 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489829114000003, 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055382103000000002, 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:59:41.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774088+0000", 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054950812000000002 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774163+0000", 2024-09-13T04:59:41.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5084000000000001e-05 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774502+0000", 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033872500000000001 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:41.240 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489807447999993, 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055600325999999999, 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:41.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774532+0000", 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055378563999999998 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774633+0000", 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100519 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774742+0000", 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010891900000000001 2024-09-13T04:59:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6094848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489791213000004, 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056713126000000003, 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:59:41.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774769+0000", 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055599061999999998 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774843+0000", 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3801999999999996e-05 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775871+0000", 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00102797 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.249 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.4897749, 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056923247000000003, 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775941+0000", 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056755141000000002 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775996+0000", 2024-09-13T04:59:41.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4580999999999998e-05 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776097+0000", 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101456 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:41.254 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.489758898999995, 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057407979999999997, 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776138+0000", 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056931362999999999 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776293+0000", 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015495600000000001 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776598+0000", 2024-09-13T04:59:41.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030449200000000001 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.479170116999995, 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053642026000000002, 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783061+0000", 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053263004000000003 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783140+0000", 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9290999999999997e-05 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783421+0000", 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028007100000000001 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.263 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.479145633000002, 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053832005000000002, 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:59:41.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783450+0000", 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053635404999999997 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783532+0000", 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1310000000000006e-05 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783635+0000", 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010324599999999999 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:41.268 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.479129665000002, 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05399698, 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:41.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783666+0000", 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053834224999999999 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783745+0000", 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8542999999999997e-05 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783816+0000", 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279999999999995e-05 2024-09-13T04:59:41.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.479096393999995, 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054125706000000003, 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:59:41.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783842+0000", 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053971225999999997 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783915+0000", 2024-09-13T04:59:41.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2657999999999995e-05 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783978+0000", 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3157000000000005e-05 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.477932862000003, 2024-09-13T04:59:41.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053864002000000001, 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:59:41.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784436+0000", 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053408069000000002 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784643+0000", 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000207171 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784880+0000", 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023680700000000001 2024-09-13T04:59:41.281 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.477900434000006, 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054149968, 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:59:41.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784958+0000", 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053896985000000001 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785046+0000", 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8321999999999999e-05 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785198+0000", 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015185 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:41.286 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.477238800999999, 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053689033999999997, 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785224+0000", 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053491874000000002 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785304+0000", 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9825000000000002e-05 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785399+0000", 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4837e-05 2024-09-13T04:59:41.290 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99151 3.f 3:f7f71ef4:::rbd_data.112afaf35b85.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.8268633439999995, 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053542113000000002, 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99151 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382083+0000", 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384133+0000", 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002049871 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384157+0000", 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3620000000000001e-05 2024-09-13T04:59:41.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384170+0000", 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3152999999999999e-05 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384238+0000", 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7883e-05 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384350+0000", 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112525 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384768+0000", 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041751899999999998 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426232+0000", 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041464366000000002 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435554+0000", 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093217539999999998 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435589+0000", 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4468000000000003e-05 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435627+0000", 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8892e-05 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.296 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99154 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.8137047479999993, 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058608128000000002, 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99154 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395242+0000", 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396241+0000", 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00099964600000000008 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396263+0000", 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1939000000000001e-05 2024-09-13T04:59:41.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396276+0000", 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2758999999999999e-05 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396371+0000", 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5233000000000001e-05 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396524+0000", 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015272899999999999 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396949+0000", 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042510200000000001 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.418937+0000", 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021988085000000001 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453775+0000", 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034838002999999999 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453813+0000", 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8180999999999999e-05 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453852+0000", 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8852e-05 2024-09-13T04:59:41.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99155 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.8126428539999999, 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057964565000000003, 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99155 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396305+0000", 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396310+0000", 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0950000000000003e-06 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396318+0000", 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.191e-06 2024-09-13T04:59:41.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396324+0000", 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.959e-06 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416450+0000", 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020125707999999999 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416531+0000", 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1093000000000004e-05 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416721+0000", 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019040800000000001 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.419351+0000", 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026298229999999999 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454195+0000", 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034844092 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454241+0000", 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5918000000000003e-05 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454270+0000", 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9391e-05 2024-09-13T04:59:41.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.309 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.309 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.309 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:41.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:41.311 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:41.306+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:41.411 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:41.406+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:41.500 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:41.500 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:41.500 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:41.500 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:41.512 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:41.506+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:41.515 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:41.612 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:41.606+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:41.681 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:41.682 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:41.682 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:41.682 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:41.682 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:41.696 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:41.738 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:41.734+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:41.794 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:41.814 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:41.810+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:41.867 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99158 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.4669864609999994, 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075195995000000002, 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:41.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99158 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397653+0000", 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400038+0000", 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023859279999999998 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400063+0000", 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4488999999999999e-05 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400077+0000", 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3698e-05 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400154+0000", 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7343999999999997e-05 2024-09-13T04:59:41.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400255+0000", 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010082299999999999 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400681+0000", 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042635500000000001 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:41.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428967+0000", 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028285346999999999 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472642+0000", 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043675249999999999 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472724+0000", 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1779000000000004e-05 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472851+0000", 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012780199999999999 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99159 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.4645361240000003, 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.076482954000000006, 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99159 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400104+0000", 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400110+0000", 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0669999999999997e-06 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400118+0000", 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4120000000000004e-06 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400123+0000", 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4979999999999997e-06 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426616+0000", 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026492418 2024-09-13T04:59:41.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426701+0000", 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4870000000000006e-05 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426882+0000", 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018111000000000001 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:41.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429464+0000", 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025825539999999999 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476516+0000", 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047051760999999998 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476551+0000", 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5317000000000001e-05 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476589+0000", 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7339999999999998e-05 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106461 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1073152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.7185610819999999, 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066606603, 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106461 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.145628+0000", 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999547 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146105+0000", 2024-09-13T04:59:41.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00047782800000000002 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146119+0000", 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193999999999999e-05 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146126+0000", 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4780000000000002e-06 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.206940+0000", 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060814147999999998 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.207024+0000", 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4049999999999997e-05 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.212687+0000", 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005662963 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.884 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106751 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1114112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.703267039, 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066917983, 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106751 2024-09-13T04:59:41.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.886 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:41.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161339+0000", 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999647 2024-09-13T04:59:41.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161379+0000", 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9650999999999997e-05 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161388+0000", 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0669999999999998e-06 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161395+0000", 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5860000000000001e-06 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.226954+0000", 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065559382999999999 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228142+0000", 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011879659999999999 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228293+0000", 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015084899999999999 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106752 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1140736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.7030620430000001, 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067202778000000005, 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106752 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161400+0000", 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998202 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161585+0000", 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018476899999999999 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161604+0000", 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9585e-05 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161612+0000", 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228406+0000", 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066793111000000002 2024-09-13T04:59:41.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228594+0000", 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188765 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228783+0000", 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018817500000000001 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106753 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.7026471770000002, 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067030303999999999, 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106753 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161619+0000", 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996243 2024-09-13T04:59:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162000+0000", 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038113399999999999 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162014+0000", 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4258e-05 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162023+0000", 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228821+0000", 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066797788999999996 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228915+0000", 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4655999999999996e-05 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229025+0000", 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010971 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106754 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1167360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.7026117359999997, 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067209986999999999, 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106754 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162029+0000", 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162033+0000", 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1659999999999998e-06 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162058+0000", 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5944000000000001e-05 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162071+0000", 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2439999999999999e-05 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229059+0000", 2024-09-13T04:59:41.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066987645999999998 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229134+0000", 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5780999999999998e-05 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229240+0000", 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105697 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106757 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1127424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.7019316389999997, 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066728500999999996, 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106757 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162079+0000", 2024-09-13T04:59:41.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9993687 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162720+0000", 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641229 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162732+0000", 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1460999999999999e-05 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162739+0000", 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.25e-06 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229267+0000", 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066528114999999999 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229346+0000", 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8710999999999999e-05 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229439+0000", 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2860000000000002e-05 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106765 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1186816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.701896327, 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066904627999999994, 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106765 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162748+0000", 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.672e-06 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162756+0000", 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8560000000000007e-06 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162792+0000", 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6309000000000001e-05 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229474+0000", 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066682166000000001 2024-09-13T04:59:41.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229551+0000", 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7187999999999993e-05 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229650+0000", 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.891e-05 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106767 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1213440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.701622983, 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066831682000000003, 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106767 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162799+0000", 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997797 2024-09-13T04:59:41.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163024+0000", 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000225336 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163037+0000", 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2870999999999999e-05 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163045+0000", 2024-09-13T04:59:41.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6329999999999996e-06 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229677+0000", 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066632620000000004 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229752+0000", 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4770000000000004e-05 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229851+0000", 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8548999999999994e-05 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106808 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1293312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6985694389999999, 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066071383999999997, 2024-09-13T04:59:41.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106808 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166075+0000", 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6450000000000001e-06 2024-09-13T04:59:41.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166080+0000", 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6949999999999997e-06 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166092+0000", 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2102e-05 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.231966+0000", 2024-09-13T04:59:41.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065873612999999998 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232047+0000", 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0848999999999998e-05 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232144+0000", 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6971000000000004e-05 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106809 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1306624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6983648169999999, 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066094452999999997, 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106809 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166099+0000", 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998217 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166282+0000", 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018357900000000001 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166294+0000", 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2074e-05 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166302+0000", 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232179+0000", 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065877145999999998 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232275+0000", 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5727e-05 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232371+0000", 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6805999999999994e-05 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106871 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1545216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6943543390000002, 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.071935387000000003, 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106871 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170290+0000", 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8370000000000001e-06 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170296+0000", 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0009999999999998e-06 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170301+0000", 2024-09-13T04:59:41.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9849999999999997e-06 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.234836+0000", 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064535151999999998 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.235101+0000", 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026450200000000001 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.242223+0000", 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0071222799999999999 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107250 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 936960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:41.933 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6696705730000003, 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066117713999999994, 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107250 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194973+0000", 2024-09-13T04:59:41.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.973e-06 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194976+0000", 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999998e-06 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194979+0000", 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9000000000000002e-06 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.259870+0000", 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064890639999999999 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.260247+0000", 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037699199999999998 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261089+0000", 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084223800000000004 2024-09-13T04:59:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107251 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 950272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6696435210000002, 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066299278000000003, 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107251 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194997+0000", 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195003+0000", 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195013+0000", 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0088999999999999e-05 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195020+0000", 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0859999999999996e-06 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261119+0000", 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066099034000000001 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261197+0000", 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7828000000000001e-05 2024-09-13T04:59:41.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261298+0000", 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100685 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107264 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.667582039, 2024-09-13T04:59:41.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066053248999999994, 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107264 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.196135+0000", 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9990754 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197065+0000", 2024-09-13T04:59:41.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093008399999999997 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197076+0000", 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0992000000000001e-05 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197083+0000", 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7059999999999998e-06 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261962+0000", 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064879036000000001 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262040+0000", 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8584999999999994e-05 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263113+0000", 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010726240000000001 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.946 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107265 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6675527150000002, 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264963999999996, 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107265 2024-09-13T04:59:41.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197091+0000", 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197099+0000", 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6459999999999993e-06 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197104+0000", 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0989999999999998e-06 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263144+0000", 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066040233000000004 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263232+0000", 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7361000000000006e-05 2024-09-13T04:59:41.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263354+0000", 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122354 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107266 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1156096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6675332249999997, 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066462531000000005, 2024-09-13T04:59:41.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107266 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197108+0000", 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197111+0000", 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.016e-06 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197119+0000", 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660000000000002e-06 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197124+0000", 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2900000000000002e-06 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263389+0000", 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264725999999996 2024-09-13T04:59:41.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263464+0000", 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5426000000000006e-05 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263571+0000", 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010699100000000001 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107414 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1699840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6565750110000002, 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069644172000000004, 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107414 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208066+0000", 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208069+0000", 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.396e-06 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208079+0000", 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0569e-05 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208085+0000", 2024-09-13T04:59:41.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6339999999999998e-06 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272576+0000", 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064490818000000005 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272673+0000", 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7427000000000001e-05 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.277711+0000", 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050376939999999997 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107622 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1315840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.6435917150000003, 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.073340916000000006, 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:41.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107622 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:41.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221052+0000", 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221059+0000", 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8770000000000004e-06 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221064+0000", 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284453+0000", 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063388694999999995 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284566+0000", 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113082 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.294391+0000", 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00982477 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:41.965 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:41.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:41.967 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:41.967 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:41.967 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:41.968 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:41.968 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:41.962+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:41.975 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:41.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:41.992 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:41.993 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:42.022 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:42.023 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:42.043 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:42.043 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:42.064 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:42.064 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:42.067 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:42.062+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:42.085 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:42.086 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:42.093 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:42.093 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:42.109 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:42.198 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:42.194+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:42.269 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:42.262+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:42.274 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:59:42.285 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:42.285 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:42.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:42.369 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:42.366+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:42.470 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:42.466+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:42.471 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:42.471 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:42.485 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:42.571 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:42.566+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:42.603 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:42.604 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:42.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:42.671 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:42.666+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:42.734 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:42.734 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:42.748 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:42.772 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:42.766+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:42.873 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:42.870+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:42.873 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:42.873 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:42.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:42.974 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:42.970+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:43.022 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:43.022 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:43.036 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:43.085 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:43.082+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:43.161 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:43.161 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:43.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:43.177 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:43.170+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:43.265 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:59:43.277 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:43.274+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:43.351 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:43.351 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:43.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:43.409 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:43.406+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:43.479 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:43.474+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:43.532 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:43.532 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:43.547 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:43.593 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:43.590+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:43.670 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:43.670 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:43.680 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:43.674+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:43.685 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:43.781 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:43.778+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:43.853 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:43.854 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:43.868 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:43.909 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:43.906+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:43.982 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:43.978+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:44.033 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:44.033 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:44.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:44.089 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:44.086+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:44.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:44.184 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:44.178+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:44.209 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:44.209 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:44.224 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:44.268 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:44.269 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:44.291 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:44.291 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:44.308 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:44.306+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:44.310 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:44.310 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:44.330 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:44.330 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:44.352 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:44.352 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:44.371 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:44.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:44.385 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:44.382+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:44.390 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:44.390 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:44.396 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596746726000006, 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.397 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T04:59:44.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.400 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.401 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596712471999993, 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T04:59:44.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T04:59:44.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.406 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596681571000005, 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.407 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:44.412 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.59662969, 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.413 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T04:59:44.417 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596597887000001, 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T04:59:44.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T04:59:44.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.422 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596563934000002, 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.423 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T04:59:44.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T04:59:44.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596516198000003, 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:44.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T04:59:44.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596476636000006, 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.432 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.433 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T04:59:44.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596428942000003, 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T04:59:44.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596365859000002, 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.441 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.445 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596329424999993, 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.449 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:44.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.451 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T04:59:44.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.453 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596295502000004, 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.454 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T04:59:44.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:44.458 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596227288999998, 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T04:59:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T04:59:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T04:59:44.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.463 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596193583000002, 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:44.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T04:59:44.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.596143390999998, 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.468 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T04:59:44.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:44.472 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.595837357999997, 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T04:59:44.476 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.595788155999998, 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.477 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T04:59:44.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.481 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.595740042000003, 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:44.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T04:59:44.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.595688565000003, 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.486 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T04:59:44.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.490 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 80.595653483999996, 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:44.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T04:59:44.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:44.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T04:59:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T04:59:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:44.495 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:44.495 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:44.496 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:44.497 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:44.498 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:44.494+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:44.498 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:44.598 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:44.594+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:44.599 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:44.599 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:44.599 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:44.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-13T04:59:44.667 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:44.668 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:44.668 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T04:59:44.668 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:44.681 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:44.725 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:44.722+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:44.800 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:44.794+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:44.848 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:44.848 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:44.848 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:44.861 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:44.905 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:44.902+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:44.913 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-09-13T04:59:44.906+0000 7f3f1af99640 -1 mon.a@0(leader).osd e101 definitely_dead 0 2024-09-13T04:59:45.003 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:45.002+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:45.032 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:45.032 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:45.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:45.032 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:45.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.032 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99147 3.8 e92/24 3:17747dba:::rbd_data.112afaf35b85.0000000000000004:head v 92'362, mlcod=92'360)", 2024-09-13T04:59:45.032 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.646873657, 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092626132, 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.033 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382240+0000", 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384691+0000", 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024508730000000001 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384738+0000", 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7345999999999997e-05 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384760+0000", 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1330000000000001e-05 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384885+0000", 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012574299999999999 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384941+0000", 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5210000000000002e-05 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474807+0000", 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089866434999999995 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474870+0000", 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2692000000000002e-05 2024-09-13T04:59:45.036 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99148 3.8 e92/24 3:17747dba:::rbd_data.112afaf35b85.0000000000000004:head v 92'364, mlcod=92'360)", 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.623511959, 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.074491979999999999, 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405603+0000", 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405621+0000", 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7927e-05 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405657+0000", 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5463999999999998e-05 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405675+0000", 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7935e-05 2024-09-13T04:59:45.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.408148+0000", 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024729909999999999 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.408388+0000", 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240292 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:45.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480055+0000", 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.071666647 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480097+0000", 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2617000000000002e-05 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99156 3.0 e92/24 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head v 92'539, mlcod=91'537)", 2024-09-13T04:59:45.041 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.619685048000001, 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.070695703999999998, 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:45.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409430+0000", 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412318+0000", 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028881380000000002 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412366+0000", 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7565000000000003e-05 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412389+0000", 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3345000000000001e-05 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412654+0000", 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000264942 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412710+0000", 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6044000000000001e-05 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480102+0000", 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.067391567999999999 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480128+0000", 2024-09-13T04:59:45.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6251e-05 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99149 3.8 e92/24 3:14f8ab77:::rbd_data.112afaf35b85.0000000000000002:head v 92'365, mlcod=92'360)", 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.61919683, 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.070251837999999997, 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409919+0000", 2024-09-13T04:59:45.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414671+0000", 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0047520030000000003 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414735+0000", 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4139000000000004e-05 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414757+0000", 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2327000000000002e-05 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.420020+0000", 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0052631450000000003 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.420067+0000", 2024-09-13T04:59:45.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6789999999999998e-05 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480133+0000", 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060066348999999998 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480172+0000", 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8757000000000002e-05 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108618 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 30720~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6507003890000007, 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05920533, 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108618 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378418+0000", 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.404e-06 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378422+0000", 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6040000000000001e-06 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378425+0000", 2024-09-13T04:59:45.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-13T04:59:45.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437429+0000", 2024-09-13T04:59:45.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059004369000000001 2024-09-13T04:59:45.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437529+0000", 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437622+0000", 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2734999999999999e-05 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108619 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 44032~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6506885110000002, 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059431339, 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108619 2024-09-13T04:59:45.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378428+0000", 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378430+0000", 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2110000000000001e-06 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378448+0000", 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7785999999999999e-05 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378462+0000", 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4098e-05 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437678+0000", 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059216116999999999 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437764+0000", 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5902999999999994e-05 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437860+0000", 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5637999999999995e-05 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.060 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108620 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 57344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6506497099999997, 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059594146000000001, 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108620 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:45.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378473+0000", 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1480000000000003e-06 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378478+0000", 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0649999999999998e-06 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378482+0000", 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2270000000000002e-06 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437899+0000", 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059417046000000001 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437977+0000", 2024-09-13T04:59:45.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8306999999999994e-05 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438062+0000", 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4769000000000001e-05 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108621 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 72704~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.065 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6506330719999998, 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059812428000000001, 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108621 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378485+0000", 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4839999999999999e-06 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378489+0000", 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7050000000000001e-06 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378493+0000", 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8970000000000001e-06 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438104+0000", 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059610677000000001 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438183+0000", 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438297+0000", 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000113187 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108622 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 86016~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6506197619999998, 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059970109000000001, 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.070 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108622 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378499+0000", 2024-09-13T04:59:45.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3080000000000002e-06 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378505+0000", 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0920000000000003e-06 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378514+0000", 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7320000000000004e-06 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438329+0000", 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059814683 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438396+0000", 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7063999999999994e-05 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438468+0000", 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1712999999999996e-05 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.074 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108623 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 99328~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6506000089999997, 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060190821999999998, 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108623 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.075 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.076 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378519+0000", 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.75e-06 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378523+0000", 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378525+0000", 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8509999999999999e-06 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438511+0000", 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059985986999999998 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438599+0000", 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7189999999999997e-05 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438708+0000", 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109392 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108632 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 209920~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:45.079 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6479388289999992, 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059361616999999998, 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108632 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381191+0000", 2024-09-13T04:59:45.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3181e-05 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381196+0000", 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7929999999999997e-06 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381199+0000", 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1820000000000002e-06 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440405+0000", 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059205525000000002 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440470+0000", 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5569999999999997e-05 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440540+0000", 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9623999999999996e-05 2024-09-13T04:59:45.083 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108633 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 223232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6479147419999993, 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059563148000000003, 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.084 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108633 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381204+0000", 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4899999999999999e-06 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381207+0000", 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5949999999999999e-06 2024-09-13T04:59:45.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381210+0000", 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9799999999999998e-06 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440582+0000", 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059371454999999997 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440666+0000", 2024-09-13T04:59:45.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4803999999999996e-05 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440766+0000", 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9130999999999997e-05 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108634 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 236544~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6479042780000004, 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059731899999999997, 2024-09-13T04:59:45.088 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108634 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.090 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381214+0000", 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.327e-06 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381218+0000", 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6100000000000002e-06 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381220+0000", 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.768e-06 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440809+0000", 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059588226000000001 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440872+0000", 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3910000000000003e-05 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440945+0000", 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2316000000000004e-05 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.092 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108635 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 249856~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6478940770000001, 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059973802, 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.093 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108635 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381224+0000", 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.406e-06 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381228+0000", 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6069999999999999e-06 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381231+0000", 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440986+0000", 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059755437000000002 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441082+0000", 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6243999999999995e-05 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441197+0000", 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114601 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108636 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 265216~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6477963809999991, 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060179980000000001, 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.097 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108636 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381234+0000", 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999132 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381323+0000", 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9089e-05 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381331+0000", 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.401e-06 2024-09-13T04:59:45.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381334+0000", 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9369999999999997e-06 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441242+0000", 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059907795999999999 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441338+0000", 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6135e-05 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441501+0000", 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016236899999999999 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108651 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 471040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:45.101 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6438376899999998, 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059311423000000002, 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108651 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385279+0000", 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385281+0000", 2024-09-13T04:59:45.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5039999999999999e-06 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385284+0000", 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7230000000000002e-06 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385287+0000", 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.084e-06 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444435+0000", 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059147995000000002 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444514+0000", 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8023999999999997e-05 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444591+0000", 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7416000000000006e-05 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.105 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108652 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 484352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6438000259999992, 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059491817000000002, 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108652 2024-09-13T04:59:45.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385319+0000", 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.804e-06 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385323+0000", 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3329999999999998e-06 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.108 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385326+0000", 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3529999999999999e-06 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444645+0000", 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059319233999999998 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444732+0000", 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6557000000000005e-05 2024-09-13T04:59:45.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444809+0000", 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6973999999999999e-05 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108669 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 194560~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6397865389999993, 2024-09-13T04:59:45.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059219301000000002, 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108669 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389330+0000", 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389335+0000", 2024-09-13T04:59:45.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9930000000000003e-06 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389339+0000", 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.07e-06 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389342+0000", 2024-09-13T04:59:45.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.897e-06 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448324+0000", 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.058981947999999999 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448430+0000", 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105351 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448550+0000", 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120301 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108670 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 196608~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6397721349999994, 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059399572999999997, 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.115 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108670 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389346+0000", 2024-09-13T04:59:45.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.466e-06 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389350+0000", 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6950000000000002e-06 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389353+0000", 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9330000000000001e-06 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448582+0000", 2024-09-13T04:59:45.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059229054000000003 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448634+0000", 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2299000000000002e-05 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448745+0000", 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110386 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108671 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 209920~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 8.6397607300000008, 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05961288, 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108671 2024-09-13T04:59:45.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389358+0000", 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.331e-06 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389361+0000", 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6820000000000001e-06 2024-09-13T04:59:45.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389364+0000", 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.994e-06 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448786+0000", 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059421307999999999 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:45.123 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448874+0000", 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8337000000000005e-05 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448969+0000", 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5486e-05 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:45.124 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:45.126 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:45.126 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:45.122+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:45.227 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:45.226+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:45.260 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:45.260 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:45.261 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:45.261 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:45.280 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:45.280 INFO:teuthology.orchestra.run.smithi001.stderr:marked out osd.1. 2024-09-13T04:59:45.294 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:59:45.354 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:45.350+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:45.409 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:45.409 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:45.409 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:45.410 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:45.410 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:45.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:45.428 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:45.422+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:45.529 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:45.522+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:45.546 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:45.546 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:45.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:45.546 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:45.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825198322000006, 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053636234999999997, 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:59:45.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772488+0000", 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053447254999999999 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772562+0000", 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4276999999999994e-05 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772656+0000", 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3993999999999995e-05 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5898240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825172929999994, 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054161726, 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772681+0000", 2024-09-13T04:59:45.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053622832000000002 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772751+0000", 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0166999999999994e-05 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773207+0000", 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045547300000000002 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5931008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825155479000003, 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054348847999999998, 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.556 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:45.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773235+0000", 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054159997000000001 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773316+0000", 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0727000000000001e-05 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773411+0000", 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5266999999999994e-05 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.560 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825138230999997, 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054777692000000003, 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:59:45.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773440+0000", 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054341649999999998 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773514+0000", 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4501999999999999e-05 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773858+0000", 2024-09-13T04:59:45.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034316700000000002 2024-09-13T04:59:45.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825115478000001, 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054958912999999998, 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:59:45.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773886+0000", 2024-09-13T04:59:45.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054770389000000003 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773964+0000", 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774062+0000", 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7059999999999996e-05 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825098527999998, 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055382103000000002, 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774088+0000", 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054950812000000002 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774163+0000", 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5084000000000001e-05 2024-09-13T04:59:45.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774502+0000", 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033872500000000001 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825076862000003, 2024-09-13T04:59:45.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055600325999999999, 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:45.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:59:45.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774532+0000", 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055378563999999998 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774633+0000", 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100519 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774742+0000", 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010891900000000001 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6094848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825060626999999, 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056713126000000003, 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.581 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774769+0000", 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055599061999999998 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774843+0000", 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3801999999999996e-05 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775871+0000", 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00102797 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.585 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825044313999996, 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056923247000000003, 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:59:45.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775941+0000", 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056755141000000002 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775996+0000", 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4580999999999998e-05 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776097+0000", 2024-09-13T04:59:45.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101456 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.825028313000004, 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057407979999999997, 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:59:45.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776138+0000", 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056931362999999999 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776293+0000", 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015495600000000001 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776598+0000", 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030449200000000001 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.594 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.814439531000005, 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053642026000000002, 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:45.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783061+0000", 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053263004000000003 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783140+0000", 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9290999999999997e-05 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783421+0000", 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028007100000000001 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.598 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.814415046999997, 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053832005000000002, 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:59:45.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783450+0000", 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053635404999999997 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783532+0000", 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1310000000000006e-05 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783635+0000", 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010324599999999999 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.814399078999998, 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05399698, 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:45.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783666+0000", 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053834224999999999 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783745+0000", 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8542999999999997e-05 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783816+0000", 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279999999999995e-05 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.607 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.814365808000005, 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054125706000000003, 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783842+0000", 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053971225999999997 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783915+0000", 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2657999999999995e-05 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783978+0000", 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3157000000000005e-05 2024-09-13T04:59:45.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.813202275999998, 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053864002000000001, 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:59:45.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784436+0000", 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053408069000000002 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784643+0000", 2024-09-13T04:59:45.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000207171 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784880+0000", 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023680700000000001 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.616 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.813169848000001, 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054149968, 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:45.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784958+0000", 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053896985000000001 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785046+0000", 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8321999999999999e-05 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785198+0000", 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015185 2024-09-13T04:59:45.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.812508214999994, 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053689033999999997, 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785224+0000", 2024-09-13T04:59:45.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053491874000000002 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785304+0000", 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9825000000000002e-05 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785399+0000", 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4837e-05 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99151 3.f 3:f7f71ef4:::rbd_data.112afaf35b85.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.162132758, 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053542113000000002, 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99151 2024-09-13T04:59:45.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:45.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382083+0000", 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384133+0000", 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002049871 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384157+0000", 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3620000000000001e-05 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384170+0000", 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3152999999999999e-05 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384238+0000", 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7883e-05 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384350+0000", 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112525 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384768+0000", 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041751899999999998 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426232+0000", 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041464366000000002 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435554+0000", 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093217539999999998 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435589+0000", 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4468000000000003e-05 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435627+0000", 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8892e-05 2024-09-13T04:59:45.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99154 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.148974162, 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058608128000000002, 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99154 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395242+0000", 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396241+0000", 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00099964600000000008 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396263+0000", 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1939000000000001e-05 2024-09-13T04:59:45.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396276+0000", 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2758999999999999e-05 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396371+0000", 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5233000000000001e-05 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396524+0000", 2024-09-13T04:59:45.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015272899999999999 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396949+0000", 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042510200000000001 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.418937+0000", 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021988085000000001 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:45.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453775+0000", 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034838002999999999 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453813+0000", 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8180999999999999e-05 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453852+0000", 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8852e-05 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99155 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.147912268000001, 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057964565000000003, 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99155 2024-09-13T04:59:45.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396305+0000", 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396310+0000", 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0950000000000003e-06 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396318+0000", 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.191e-06 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396324+0000", 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.959e-06 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416450+0000", 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020125707999999999 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416531+0000", 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1093000000000004e-05 2024-09-13T04:59:45.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416721+0000", 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019040800000000001 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.419351+0000", 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026298229999999999 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454195+0000", 2024-09-13T04:59:45.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034844092 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454241+0000", 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5918000000000003e-05 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454270+0000", 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9391e-05 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:45.643 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:45.644 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:45.645 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:45.645 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:45.642+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:45.746 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:45.742+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:45.818 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:45.818 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:45.818 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:45.818 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:45.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:45.847 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:45.842+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:45.948 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:45.942+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:45.996 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:45.996 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:45.996 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:45.997 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:45.997 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:46.012 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:46.062 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:46.058+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:46.140 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:46.140 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:46.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99158 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.741597925000001, 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075195995000000002, 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99158 2024-09-13T04:59:46.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397653+0000", 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400038+0000", 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023859279999999998 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400063+0000", 2024-09-13T04:59:46.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4488999999999999e-05 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400077+0000", 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3698e-05 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400154+0000", 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7343999999999997e-05 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400255+0000", 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010082299999999999 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400681+0000", 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042635500000000001 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:46.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428967+0000", 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028285346999999999 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472642+0000", 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043675249999999999 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:46.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472724+0000", 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1779000000000004e-05 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472851+0000", 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012780199999999999 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99159 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:46.147 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.739147588, 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.076482954000000006, 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99159 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400104+0000", 2024-09-13T04:59:46.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400110+0000", 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0669999999999997e-06 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400118+0000", 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4120000000000004e-06 2024-09-13T04:59:46.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400123+0000", 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4979999999999997e-06 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426616+0000", 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026492418 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426701+0000", 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4870000000000006e-05 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426882+0000", 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018111000000000001 2024-09-13T04:59:46.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429464+0000", 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025825539999999999 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476516+0000", 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047051760999999998 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476551+0000", 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5317000000000001e-05 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476589+0000", 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7339999999999998e-05 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106461 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1073152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9931725460000003, 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066606603, 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106461 2024-09-13T04:59:46.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.156 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:46.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:46.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.145628+0000", 2024-09-13T04:59:46.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999547 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146105+0000", 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00047782800000000002 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146119+0000", 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193999999999999e-05 2024-09-13T04:59:46.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146126+0000", 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4780000000000002e-06 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.206940+0000", 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060814147999999998 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.207024+0000", 2024-09-13T04:59:46.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4049999999999997e-05 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.212687+0000", 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005662963 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106751 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1114112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:46.160 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9778785029999995, 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066917983, 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106751 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:46.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161339+0000", 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999647 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161379+0000", 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9650999999999997e-05 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161388+0000", 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0669999999999998e-06 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161395+0000", 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5860000000000001e-06 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.226954+0000", 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065559382999999999 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228142+0000", 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011879659999999999 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228293+0000", 2024-09-13T04:59:46.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015084899999999999 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106752 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1140736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9776735070000004, 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067202778000000005, 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106752 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161400+0000", 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998202 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161585+0000", 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018476899999999999 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161604+0000", 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9585e-05 2024-09-13T04:59:46.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161612+0000", 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228406+0000", 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066793111000000002 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228594+0000", 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188765 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228783+0000", 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018817500000000001 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106753 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.169 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9772586410000006, 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067030303999999999, 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106753 2024-09-13T04:59:46.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.171 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:46.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161619+0000", 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996243 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162000+0000", 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038113399999999999 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162014+0000", 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4258e-05 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162023+0000", 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228821+0000", 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066797788999999996 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228915+0000", 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4655999999999996e-05 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229025+0000", 2024-09-13T04:59:46.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010971 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106754 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1167360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9772231999999992, 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067209986999999999, 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106754 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162029+0000", 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162033+0000", 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1659999999999998e-06 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162058+0000", 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5944000000000001e-05 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162071+0000", 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2439999999999999e-05 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229059+0000", 2024-09-13T04:59:46.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066987645999999998 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229134+0000", 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5780999999999998e-05 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229240+0000", 2024-09-13T04:59:46.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105697 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106757 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1127424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9765431030000009, 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066728500999999996, 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106757 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162079+0000", 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9993687 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162720+0000", 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641229 2024-09-13T04:59:46.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162732+0000", 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1460999999999999e-05 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162739+0000", 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.25e-06 2024-09-13T04:59:46.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229267+0000", 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066528114999999999 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229346+0000", 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8710999999999999e-05 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229439+0000", 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2860000000000002e-05 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106765 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1186816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9765077909999995, 2024-09-13T04:59:46.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066904627999999994, 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106765 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:46.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162748+0000", 2024-09-13T04:59:46.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.672e-06 2024-09-13T04:59:46.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162756+0000", 2024-09-13T04:59:46.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8560000000000007e-06 2024-09-13T04:59:46.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162792+0000", 2024-09-13T04:59:46.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6309000000000001e-05 2024-09-13T04:59:46.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229474+0000", 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066682166000000001 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229551+0000", 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7187999999999993e-05 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229650+0000", 2024-09-13T04:59:46.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.891e-05 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106767 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1213440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9762344469999995, 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066831682000000003, 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106767 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162799+0000", 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997797 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163024+0000", 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000225336 2024-09-13T04:59:46.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163037+0000", 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2870999999999999e-05 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163045+0000", 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6329999999999996e-06 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229677+0000", 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066632620000000004 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229752+0000", 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4770000000000004e-05 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229851+0000", 2024-09-13T04:59:46.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8548999999999994e-05 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106808 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1293312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9731809029999994, 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066071383999999997, 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.196 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106808 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:46.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166075+0000", 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6450000000000001e-06 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166080+0000", 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6949999999999997e-06 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166092+0000", 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2102e-05 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.231966+0000", 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065873612999999998 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232047+0000", 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0848999999999998e-05 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232144+0000", 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6971000000000004e-05 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106809 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1306624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9729762809999993, 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066094452999999997, 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106809 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166099+0000", 2024-09-13T04:59:46.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998217 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166282+0000", 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018357900000000001 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166294+0000", 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2074e-05 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166302+0000", 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232179+0000", 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065877145999999998 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232275+0000", 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5727e-05 2024-09-13T04:59:46.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232371+0000", 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6805999999999994e-05 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106871 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1545216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9689658029999997, 2024-09-13T04:59:46.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.071935387000000003, 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106871 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170290+0000", 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8370000000000001e-06 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170296+0000", 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0009999999999998e-06 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170301+0000", 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9849999999999997e-06 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.234836+0000", 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064535151999999998 2024-09-13T04:59:46.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.235101+0000", 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026450200000000001 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.242223+0000", 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0071222799999999999 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.209 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107250 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 936960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9442820370000007, 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066117713999999994, 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107250 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194973+0000", 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.973e-06 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194976+0000", 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999998e-06 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194979+0000", 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9000000000000002e-06 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.259870+0000", 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064890639999999999 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.260247+0000", 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037699199999999998 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261089+0000", 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084223800000000004 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107251 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 950272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9442549850000006, 2024-09-13T04:59:46.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066299278000000003, 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107251 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194997+0000", 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195003+0000", 2024-09-13T04:59:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195013+0000", 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0088999999999999e-05 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195020+0000", 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0859999999999996e-06 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261119+0000", 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066099034000000001 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261197+0000", 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7828000000000001e-05 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261298+0000", 2024-09-13T04:59:46.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100685 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107264 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9421935030000004, 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066053248999999994, 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.219 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107264 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.196135+0000", 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9990754 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197065+0000", 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093008399999999997 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197076+0000", 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0992000000000001e-05 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197083+0000", 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7059999999999998e-06 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261962+0000", 2024-09-13T04:59:46.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064879036000000001 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262040+0000", 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8584999999999994e-05 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263113+0000", 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010726240000000001 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107265 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9421641790000006, 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264963999999996, 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107265 2024-09-13T04:59:46.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197091+0000", 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197099+0000", 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6459999999999993e-06 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197104+0000", 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0989999999999998e-06 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263144+0000", 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066040233000000004 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263232+0000", 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7361000000000006e-05 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263354+0000", 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122354 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107266 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1156096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9421446889999991, 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066462531000000005, 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107266 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.229 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197108+0000", 2024-09-13T04:59:46.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197111+0000", 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.016e-06 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197119+0000", 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660000000000002e-06 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197124+0000", 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2900000000000002e-06 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263389+0000", 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264725999999996 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263464+0000", 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5426000000000006e-05 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263571+0000", 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010699100000000001 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107414 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1699840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:46.233 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9311864750000005, 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069644172000000004, 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107414 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208066+0000", 2024-09-13T04:59:46.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208069+0000", 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.396e-06 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208079+0000", 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0569e-05 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208085+0000", 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6339999999999998e-06 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272576+0000", 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064490818000000005 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272673+0000", 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7427000000000001e-05 2024-09-13T04:59:46.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.277711+0000", 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050376939999999997 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107622 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1315840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9182031790000007, 2024-09-13T04:59:46.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.073340916000000006, 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107622 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221052+0000", 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-13T04:59:46.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221059+0000", 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8770000000000004e-06 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221064+0000", 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:46.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284453+0000", 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063388694999999995 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284566+0000", 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113082 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:46.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:46.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.294391+0000", 2024-09-13T04:59:46.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00982477 2024-09-13T04:59:46.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:46.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:46.243 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:46.245 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:46.245 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:46.242+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:46.281 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:59:46.345 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:46.342+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:46.410 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:46.410 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:46.411 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:46.411 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:46.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:46.470 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:46.466+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:46.499 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:46.550 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:46.546+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:46.551 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:46.551 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:46.551 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:46.551 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:46.551 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:46.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:46.643 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:46.644 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:46.648 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:46.642+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:46.661 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:46.661 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:46.678 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:46.678 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:46.695 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:46.715 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:46.715 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:46.728 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:46.728 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:46.735 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:46.736 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:46.743 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:46.749 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:46.746+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:46.756 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:46.778 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:46.851 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:46.846+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:46.874 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:46.875 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:46.889 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:46.952 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:46.946+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:47.012 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:47.012 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:47.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:47.052 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:47.046+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:47.146 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:47.146 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:47.152 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:47.146+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:47.162 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:47.253 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:47.250+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:47.302 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:47.302 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:47.325 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:47.369 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:47.366+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:47.455 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:47.450+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:47.466 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:47.466 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:47.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:47.556 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:47.550+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:47.657 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:47.650+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:47.668 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:47.668 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:47.689 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:47.758 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:47.753+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:47.834 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:47.834 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:47.852 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:47.859 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:47.854+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:47.959 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:47.953+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:48.025 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:48.025 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:48.040 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:48.060 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:48.054+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:48.161 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:48.157+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:48.209 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:48.209 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:48.224 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:48.263 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:48.258+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:48.311 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T04:59:48.363 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:48.357+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:48.408 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:48.408 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:48.423 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:48.465 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:48.461+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:48.564 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:48.561+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:48.590 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:48.591 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:48.606 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:48.665 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:48.662+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:48.766 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:48.761+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:48.768 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:48.769 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:48.783 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:48.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:48.869 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:48.865+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T04:59:48.960 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:48.960 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:48.968 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:48.961+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:48.976 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:48.977 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:48.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:48.994 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:48.994 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:49.013 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:49.013 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:49.030 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:49.049 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:49.049 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:49.069 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:49.062+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:49.070 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:49.070 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:49.089 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:49.089 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:49.109 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:49.109 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:49.146 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:49.147 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T04:59:49.147 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T04:59:49.147 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T04:59:49.147 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:49.161 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:49.173 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:49.170+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:49.270 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:49.266+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:49.285 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99147 3.8 e92/24 3:17747dba:::rbd_data.112afaf35b85.0000000000000004:head v 92'362, mlcod=92'360)", 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.900224464000001, 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.092626132, 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382244+0000", 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.382240+0000", 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999967 2024-09-13T04:59:49.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384691+0000", 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024508730000000001 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384738+0000", 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7345999999999997e-05 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384760+0000", 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1330000000000001e-05 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384885+0000", 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012574299999999999 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.384941+0000", 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5210000000000002e-05 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.289 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474807+0000", 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.089866434999999995 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.474870+0000", 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2692000000000002e-05 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.290 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99148 3.8 e92/24 3:17747dba:::rbd_data.112afaf35b85.0000000000000004:head v 92'364, mlcod=92'360)", 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.876862765999999, 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.074491979999999999, 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405605+0000", 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405603+0000", 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405621+0000", 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7927e-05 2024-09-13T04:59:49.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405657+0000", 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5463999999999998e-05 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405675+0000", 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7935e-05 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.408148+0000", 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0024729909999999999 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.408388+0000", 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000240292 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480055+0000", 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.071666647 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.294 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480097+0000", 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2617000000000002e-05 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99156 3.0 e92/24 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head v 92'539, mlcod=91'537)", 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.873035855000001, 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.070695703999999998, 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409432+0000", 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409430+0000", 2024-09-13T04:59:49.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412318+0000", 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0028881380000000002 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412366+0000", 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7565000000000003e-05 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412389+0000", 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3345000000000001e-05 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412654+0000", 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000264942 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.412710+0000", 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6044000000000001e-05 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480102+0000", 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.067391567999999999 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480128+0000", 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6251e-05 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop(client.4394.0:99149 3.8 e92/24 3:14f8ab77:::rbd_data.112afaf35b85.0000000000000002:head v 92'365, mlcod=92'360)", 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 16.872547637, 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.070251837999999997, 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409920+0000", 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.409919+0000", 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414671+0000", 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0047520030000000003 2024-09-13T04:59:49.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414735+0000", 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4139000000000004e-05 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.414757+0000", 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2327000000000002e-05 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.420020+0000", 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0052631450000000003 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.420067+0000", 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6789999999999998e-05 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480133+0000", 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060066348999999998 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.303 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480172+0000", 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8757000000000002e-05 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108618 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 30720~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.904051195999999, 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05920533, 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.304 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108618 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378417+0000", 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378418+0000", 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.404e-06 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378422+0000", 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6040000000000001e-06 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378425+0000", 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.0299999999999998e-06 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437429+0000", 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059004369000000001 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437529+0000", 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437622+0000", 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2734999999999999e-05 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.308 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108619 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 44032~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.904039318000001, 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059431339, 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108619 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378429+0000", 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378428+0000", 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378430+0000", 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2110000000000001e-06 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378448+0000", 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.7785999999999999e-05 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378462+0000", 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4098e-05 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437678+0000", 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059216116999999999 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437764+0000", 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5902999999999994e-05 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437860+0000", 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5637999999999995e-05 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.312 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108620 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 57344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.904000517, 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059594146000000001, 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108620 2024-09-13T04:59:49.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378467+0000", 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378473+0000", 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1480000000000003e-06 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378478+0000", 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0649999999999998e-06 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378482+0000", 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2270000000000002e-06 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437899+0000", 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059417046000000001 2024-09-13T04:59:49.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.437977+0000", 2024-09-13T04:59:49.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8306999999999994e-05 2024-09-13T04:59:49.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438062+0000", 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4769000000000001e-05 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108621 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 72704~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.903983879, 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059812428000000001, 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108621 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378484+0000", 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378485+0000", 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4839999999999999e-06 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378489+0000", 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7050000000000001e-06 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378493+0000", 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8970000000000001e-06 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438104+0000", 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059610677000000001 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438183+0000", 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9740999999999994e-05 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438297+0000", 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000113187 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108622 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 86016~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:49.322 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.903970569, 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059970109000000001, 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108622 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378497+0000", 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378499+0000", 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.3080000000000002e-06 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378505+0000", 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0920000000000003e-06 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378514+0000", 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7320000000000004e-06 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438329+0000", 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059814683 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438396+0000", 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.7063999999999994e-05 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438468+0000", 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1712999999999996e-05 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.326 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108623 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 99328~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.903950816, 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060190821999999998, 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108623 2024-09-13T04:59:49.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.328 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378517+0000", 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378519+0000", 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.75e-06 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378523+0000", 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9600000000000002e-06 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.378525+0000", 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.8509999999999999e-06 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438511+0000", 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059985986999999998 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438599+0000", 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.7189999999999997e-05 2024-09-13T04:59:49.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.438708+0000", 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000109392 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108632 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 209920~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.901289636, 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059361616999999998, 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.331 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108632 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381178+0000", 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381191+0000", 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3181e-05 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.333 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381196+0000", 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7929999999999997e-06 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381199+0000", 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1820000000000002e-06 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440405+0000", 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059205525000000002 2024-09-13T04:59:49.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440470+0000", 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5569999999999997e-05 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440540+0000", 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9623999999999996e-05 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.335 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108633 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 223232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.901265549, 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059563148000000003, 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108633 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381202+0000", 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381204+0000", 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4899999999999999e-06 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381207+0000", 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.5949999999999999e-06 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381210+0000", 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9799999999999998e-06 2024-09-13T04:59:49.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440582+0000", 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059371454999999997 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440666+0000", 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4803999999999996e-05 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440766+0000", 2024-09-13T04:59:49.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9130999999999997e-05 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108634 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 236544~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.901255085000001, 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059731899999999997, 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.340 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108634 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381213+0000", 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381214+0000", 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.327e-06 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381218+0000", 2024-09-13T04:59:49.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6100000000000002e-06 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381220+0000", 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.768e-06 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440809+0000", 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059588226000000001 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440872+0000", 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3910000000000003e-05 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440945+0000", 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2316000000000004e-05 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.344 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108635 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 249856~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.901244884, 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059973802, 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108635 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381223+0000", 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381224+0000", 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.406e-06 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381228+0000", 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6069999999999999e-06 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381231+0000", 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.7700000000000002e-06 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.440986+0000", 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059755437000000002 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441082+0000", 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6243999999999995e-05 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441197+0000", 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000114601 2024-09-13T04:59:49.348 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108636 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 265216~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.901147187999999, 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060179980000000001, 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.349 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108636 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381321+0000", 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381234+0000", 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999132 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381323+0000", 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9089e-05 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381331+0000", 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.401e-06 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.381334+0000", 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9369999999999997e-06 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441242+0000", 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059907795999999999 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441338+0000", 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.6135e-05 2024-09-13T04:59:49.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.441501+0000", 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016236899999999999 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108651 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 471040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.897188497, 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059311423000000002, 2024-09-13T04:59:49.353 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108651 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385280+0000", 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385279+0000", 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385281+0000", 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5039999999999999e-06 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385284+0000", 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.7230000000000002e-06 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385287+0000", 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.084e-06 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444435+0000", 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059147995000000002 2024-09-13T04:59:49.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444514+0000", 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8023999999999997e-05 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444591+0000", 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7416000000000006e-05 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108652 3.b 3:d79cc19d:::rbd_data.112a28ed7f25.0000000000000005:2a [read 484352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.897150833, 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059491817000000002, 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108652 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385317+0000", 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385319+0000", 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.804e-06 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385323+0000", 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3329999999999998e-06 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.385326+0000", 2024-09-13T04:59:49.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3529999999999999e-06 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444645+0000", 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059319233999999998 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444732+0000", 2024-09-13T04:59:49.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.6557000000000005e-05 2024-09-13T04:59:49.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.444809+0000", 2024-09-13T04:59:49.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6973999999999999e-05 2024-09-13T04:59:49.364 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.364 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108669 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 194560~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.893137346, 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059219301000000002, 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108669 2024-09-13T04:59:49.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389331+0000", 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389330+0000", 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389335+0000", 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9930000000000003e-06 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389339+0000", 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.07e-06 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389342+0000", 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.897e-06 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448324+0000", 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.058981947999999999 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448430+0000", 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000105351 2024-09-13T04:59:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448550+0000", 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000120301 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108670 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 196608~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.893122942, 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059399572999999997, 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.369 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108670 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389345+0000", 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389346+0000", 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.466e-06 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389350+0000", 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6950000000000002e-06 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389353+0000", 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9330000000000001e-06 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448582+0000", 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059229054000000003 2024-09-13T04:59:49.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448634+0000", 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2299000000000002e-05 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448745+0000", 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000110386 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.373 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:108671 3.b 3:d2d25a17:::rbd_data.112a28ed7f25.0000000000000004:2a [read 209920~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 12.893111536999999, 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.05961288, 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 108671 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T04:59:49.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389356+0000", 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389358+0000", 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.331e-06 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389361+0000", 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6820000000000001e-06 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.389364+0000", 2024-09-13T04:59:49.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.994e-06 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448786+0000", 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.059421307999999999 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448874+0000", 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8337000000000005e-05 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T04:59:49.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T04:59:49.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:36.448969+0000", 2024-09-13T04:59:49.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5486e-05 2024-09-13T04:59:49.378 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.378 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.378 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.378 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T04:59:49.378 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T04:59:49.378 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T04:59:49.379 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:49.373+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:49.380 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:49.481 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:49.477+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:49.514 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:49.514 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:49.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:49.582 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:49.578+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:49.655 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:49.656 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:49.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:49.683 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:49.677+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:49.784 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:49.777+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:49.845 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122868003999997, 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053636234999999997, 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:59:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772488+0000", 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053447254999999999 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772562+0000", 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4276999999999994e-05 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772656+0000", 2024-09-13T04:59:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3993999999999995e-05 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5898240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122842611999999, 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054161726, 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772681+0000", 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053622832000000002 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772751+0000", 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0166999999999994e-05 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773207+0000", 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045547300000000002 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5931008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122825160999994, 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054348847999999998, 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:59:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773235+0000", 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054159997000000001 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773316+0000", 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0727000000000001e-05 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773411+0000", 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5266999999999994e-05 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122807913000003, 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054777692000000003, 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:59:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773440+0000", 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054341649999999998 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773514+0000", 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4501999999999999e-05 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773858+0000", 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034316700000000002 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122785160000006, 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054958912999999998, 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773886+0000", 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054770389000000003 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773964+0000", 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774062+0000", 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7059999999999996e-05 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122768210000004, 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055382103000000002, 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774088+0000", 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054950812000000002 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774163+0000", 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5084000000000001e-05 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774502+0000", 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033872500000000001 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122746543999995, 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055600325999999999, 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774532+0000", 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055378563999999998 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774633+0000", 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100519 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774742+0000", 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010891900000000001 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6094848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122730309000005, 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056713126000000003, 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774769+0000", 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055599061999999998 2024-09-13T04:59:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774843+0000", 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3801999999999996e-05 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775871+0000", 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00102797 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122713996000002, 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056923247000000003, 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:59:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775941+0000", 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056755141000000002 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775996+0000", 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4580999999999998e-05 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776097+0000", 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101456 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.122697994999996, 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057407979999999997, 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776138+0000", 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056931362999999999 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776293+0000", 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015495600000000001 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776598+0000", 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030449200000000001 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.112109212999997, 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053642026000000002, 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:59:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783061+0000", 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053263004000000003 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783140+0000", 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9290999999999997e-05 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783421+0000", 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028007100000000001 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.112084729000003, 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053832005000000002, 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783450+0000", 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053635404999999997 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783532+0000", 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1310000000000006e-05 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783635+0000", 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010324599999999999 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.112068761000003, 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05399698, 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:59:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783666+0000", 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053834224999999999 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783745+0000", 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8542999999999997e-05 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783816+0000", 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279999999999995e-05 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.112035489999997, 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054125706000000003, 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:59:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783842+0000", 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053971225999999997 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783915+0000", 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2657999999999995e-05 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783978+0000", 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3157000000000005e-05 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.110871958000004, 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053864002000000001, 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:59:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784436+0000", 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053408069000000002 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784643+0000", 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000207171 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784880+0000", 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023680700000000001 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.110839530000007, 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054149968, 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:59:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784958+0000", 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053896985000000001 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785046+0000", 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8321999999999999e-05 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785198+0000", 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015185 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.110177897, 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053689033999999997, 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:59:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785224+0000", 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053491874000000002 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785304+0000", 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9825000000000002e-05 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785399+0000", 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4837e-05 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99151 3.f 3:f7f71ef4:::rbd_data.112afaf35b85.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.459802440000001, 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053542113000000002, 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99151 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382083+0000", 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384133+0000", 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002049871 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384157+0000", 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3620000000000001e-05 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384170+0000", 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3152999999999999e-05 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384238+0000", 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7883e-05 2024-09-13T04:59:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384350+0000", 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112525 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384768+0000", 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041751899999999998 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426232+0000", 2024-09-13T04:59:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041464366000000002 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435554+0000", 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093217539999999998 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435589+0000", 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4468000000000003e-05 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435627+0000", 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8892e-05 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99154 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.446643844, 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058608128000000002, 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99154 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395242+0000", 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396241+0000", 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00099964600000000008 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396263+0000", 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1939000000000001e-05 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396276+0000", 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2758999999999999e-05 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396371+0000", 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5233000000000001e-05 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396524+0000", 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015272899999999999 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396949+0000", 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042510200000000001 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.418937+0000", 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021988085000000001 2024-09-13T04:59:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453775+0000", 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034838002999999999 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453813+0000", 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8180999999999999e-05 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453852+0000", 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8852e-05 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99155 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.445581950000001, 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057964565000000003, 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99155 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396305+0000", 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396310+0000", 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0950000000000003e-06 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396318+0000", 2024-09-13T04:59:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.191e-06 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396324+0000", 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.959e-06 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416450+0000", 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020125707999999999 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416531+0000", 2024-09-13T04:59:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1093000000000004e-05 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416721+0000", 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019040800000000001 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.419351+0000", 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026298229999999999 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454195+0000", 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034844092 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454241+0000", 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5918000000000003e-05 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454270+0000", 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9391e-05 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:49.939 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:49.940 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:49.941 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:49.933+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:50.041 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:50.034+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:50.081 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:50.081 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:50.081 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:50.081 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:50.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:50.182 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:50.178+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:50.243 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:50.238+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:50.260 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:50.260 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:50.260 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:50.260 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:50.260 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:50.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:50.309 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:50.309 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:50.309 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T04:59:50.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-13T04:59:50.362 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:50.358+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:50.443 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:50.443 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99158 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.042845896999999, 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075195995000000002, 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99158 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397653+0000", 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400038+0000", 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023859279999999998 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400063+0000", 2024-09-13T04:59:50.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4488999999999999e-05 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400077+0000", 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3698e-05 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400154+0000", 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7343999999999997e-05 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400255+0000", 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010082299999999999 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400681+0000", 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042635500000000001 2024-09-13T04:59:50.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428967+0000", 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028285346999999999 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472642+0000", 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043675249999999999 2024-09-13T04:59:50.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472724+0000", 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1779000000000004e-05 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472851+0000", 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012780199999999999 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99159 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.04039556, 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.076482954000000006, 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99159 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.451 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400104+0000", 2024-09-13T04:59:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400110+0000", 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0669999999999997e-06 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400118+0000", 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4120000000000004e-06 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400123+0000", 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4979999999999997e-06 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426616+0000", 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026492418 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426701+0000", 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4870000000000006e-05 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426882+0000", 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018111000000000001 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429464+0000", 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025825539999999999 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476516+0000", 2024-09-13T04:59:50.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047051760999999998 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476551+0000", 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5317000000000001e-05 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476589+0000", 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7339999999999998e-05 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106461 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1073152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.294420518000001, 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066606603, 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106461 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.457 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.145628+0000", 2024-09-13T04:59:50.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999547 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146105+0000", 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00047782800000000002 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146119+0000", 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193999999999999e-05 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146126+0000", 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4780000000000002e-06 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.206940+0000", 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060814147999999998 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.207024+0000", 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4049999999999997e-05 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.212687+0000", 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005662963 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106751 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1114112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.279126475, 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066917983, 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.461 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106751 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161339+0000", 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999647 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161379+0000", 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9650999999999997e-05 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161388+0000", 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0669999999999998e-06 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161395+0000", 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5860000000000001e-06 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.226954+0000", 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065559382999999999 2024-09-13T04:59:50.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228142+0000", 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011879659999999999 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228293+0000", 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015084899999999999 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106752 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1140736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.278921478999999, 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067202778000000005, 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106752 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161400+0000", 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998202 2024-09-13T04:59:50.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161585+0000", 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018476899999999999 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161604+0000", 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9585e-05 2024-09-13T04:59:50.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161612+0000", 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228406+0000", 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066793111000000002 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228594+0000", 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188765 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228783+0000", 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018817500000000001 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106753 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.470 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.278506612999999, 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067030303999999999, 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106753 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161619+0000", 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996243 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162000+0000", 2024-09-13T04:59:50.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038113399999999999 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162014+0000", 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4258e-05 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162023+0000", 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228821+0000", 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066797788999999996 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228915+0000", 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4655999999999996e-05 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229025+0000", 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010971 2024-09-13T04:59:50.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106754 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1167360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.278471172, 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067209986999999999, 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106754 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162029+0000", 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162033+0000", 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1659999999999998e-06 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162058+0000", 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5944000000000001e-05 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162071+0000", 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2439999999999999e-05 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229059+0000", 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066987645999999998 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229134+0000", 2024-09-13T04:59:50.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5780999999999998e-05 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229240+0000", 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105697 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106757 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1127424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.277791075, 2024-09-13T04:59:50.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066728500999999996, 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106757 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162079+0000", 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9993687 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162720+0000", 2024-09-13T04:59:50.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641229 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162732+0000", 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1460999999999999e-05 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162739+0000", 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.25e-06 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229267+0000", 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066528114999999999 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229346+0000", 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8710999999999999e-05 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229439+0000", 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2860000000000002e-05 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106765 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1186816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.277755763, 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066904627999999994, 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106765 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.484 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162748+0000", 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.672e-06 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162756+0000", 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8560000000000007e-06 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162792+0000", 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6309000000000001e-05 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229474+0000", 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066682166000000001 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229551+0000", 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7187999999999993e-05 2024-09-13T04:59:50.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229650+0000", 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.891e-05 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106767 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1213440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.277482419, 2024-09-13T04:59:50.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066831682000000003, 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106767 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162799+0000", 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997797 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163024+0000", 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000225336 2024-09-13T04:59:50.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163037+0000", 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2870999999999999e-05 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163045+0000", 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6329999999999996e-06 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229677+0000", 2024-09-13T04:59:50.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066632620000000004 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229752+0000", 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4770000000000004e-05 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229851+0000", 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8548999999999994e-05 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106808 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1293312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.274428875, 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066071383999999997, 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106808 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166075+0000", 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6450000000000001e-06 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166080+0000", 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6949999999999997e-06 2024-09-13T04:59:50.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166092+0000", 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2102e-05 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.231966+0000", 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065873612999999998 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232047+0000", 2024-09-13T04:59:50.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0848999999999998e-05 2024-09-13T04:59:50.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232144+0000", 2024-09-13T04:59:50.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6971000000000004e-05 2024-09-13T04:59:50.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.497 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106809 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1306624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.274224253, 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066094452999999997, 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106809 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166099+0000", 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998217 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166282+0000", 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018357900000000001 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166294+0000", 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2074e-05 2024-09-13T04:59:50.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166302+0000", 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232179+0000", 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065877145999999998 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232275+0000", 2024-09-13T04:59:50.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5727e-05 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232371+0000", 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6805999999999994e-05 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106871 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1545216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:50.502 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.270213775, 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.071935387000000003, 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106871 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170290+0000", 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8370000000000001e-06 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170296+0000", 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0009999999999998e-06 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170301+0000", 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9849999999999997e-06 2024-09-13T04:59:50.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.234836+0000", 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064535151999999998 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.235101+0000", 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026450200000000001 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.242223+0000", 2024-09-13T04:59:50.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0071222799999999999 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107250 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 936960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.245530008999999, 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066117713999999994, 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107250 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194973+0000", 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.973e-06 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194976+0000", 2024-09-13T04:59:50.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999998e-06 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194979+0000", 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9000000000000002e-06 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.259870+0000", 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064890639999999999 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.260247+0000", 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037699199999999998 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261089+0000", 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084223800000000004 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107251 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 950272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.245502956999999, 2024-09-13T04:59:50.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066299278000000003, 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107251 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194997+0000", 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195003+0000", 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:50.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195013+0000", 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0088999999999999e-05 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195020+0000", 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0859999999999996e-06 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261119+0000", 2024-09-13T04:59:50.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066099034000000001 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261197+0000", 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7828000000000001e-05 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261298+0000", 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100685 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107264 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.243441474999999, 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066053248999999994, 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107264 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.516 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.196135+0000", 2024-09-13T04:59:50.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9990754 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197065+0000", 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093008399999999997 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197076+0000", 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0992000000000001e-05 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197083+0000", 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7059999999999998e-06 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261962+0000", 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064879036000000001 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262040+0000", 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8584999999999994e-05 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263113+0000", 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010726240000000001 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107265 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.243412150999999, 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264963999999996, 2024-09-13T04:59:50.520 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107265 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197091+0000", 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-13T04:59:50.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197099+0000", 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6459999999999993e-06 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197104+0000", 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0989999999999998e-06 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263144+0000", 2024-09-13T04:59:50.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066040233000000004 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263232+0000", 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7361000000000006e-05 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263354+0000", 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122354 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107266 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1156096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.243392661, 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066462531000000005, 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107266 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197108+0000", 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197111+0000", 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.016e-06 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197119+0000", 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660000000000002e-06 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197124+0000", 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2900000000000002e-06 2024-09-13T04:59:50.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263389+0000", 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264725999999996 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263464+0000", 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5426000000000006e-05 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263571+0000", 2024-09-13T04:59:50.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010699100000000001 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107414 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1699840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.232434446999999, 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069644172000000004, 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107414 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:50.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208066+0000", 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208069+0000", 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.396e-06 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208079+0000", 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0569e-05 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208085+0000", 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6339999999999998e-06 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272576+0000", 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064490818000000005 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272673+0000", 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7427000000000001e-05 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.277711+0000", 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050376939999999997 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.533 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107622 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1315840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.219451150999999, 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.073340916000000006, 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107622 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:50.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:50.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221052+0000", 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221059+0000", 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8770000000000004e-06 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221064+0000", 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284453+0000", 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063388694999999995 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284566+0000", 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113082 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.294391+0000", 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00982477 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:50.538 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:50.538 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:50.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:50.540 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T04:59:50.540 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:50.533+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:50.640 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:50.634+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:50.704 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:50.704 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:50.704 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:50.704 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:50.719 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:50.741 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:50.737+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:50.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-13T04:59:50.842 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:50.837+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:50.913 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:50.913 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:50.913 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:50.913 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:50.913 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:50.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:50.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T04:59:50.973 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T04:59:50.973 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:50.966+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:51.057 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:51.054+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.605832583999998, 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:59:51.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:59:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103023 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.272303798999999, 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042297819, 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.074 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103023 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794689+0000", 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7060000000000001e-06 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794693+0000", 2024-09-13T04:59:51.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.811e-06 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794696+0000", 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1429999999999998e-06 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836839+0000", 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042142577000000001 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836920+0000", 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1012000000000003e-05 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836986+0000", 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5841999999999997e-05 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.078 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103024 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 482304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.272292931999999, 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042470763000000002, 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103024 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794698+0000", 2024-09-13T04:59:51.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794700+0000", 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.567e-06 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794704+0000", 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8399999999999997e-06 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794707+0000", 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1209999999999998e-06 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837011+0000", 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042303828000000002 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837096+0000", 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5148999999999994e-05 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837169+0000", 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3521000000000003e-05 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103025 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.2722821, 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042586098000000003, 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103025 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794709+0000", 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794711+0000", 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5999999999999999e-06 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794715+0000", 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8260000000000003e-06 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794718+0000", 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837196+0000", 2024-09-13T04:59:51.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042477695000000003 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837252+0000", 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6808000000000002e-05 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837296+0000", 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3327e-05 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103026 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 508928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.272271422999999, 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042722826999999998, 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103026 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.088 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794721+0000", 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4619999999999999e-06 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794725+0000", 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7890000000000001e-06 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794736+0000", 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1202999999999999e-05 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837322+0000", 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042585852 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837386+0000", 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3942000000000005e-05 2024-09-13T04:59:51.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837443+0000", 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6841e-05 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103027 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.272252119999999, 2024-09-13T04:59:51.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042880907000000003, 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103027 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794739+0000", 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794741+0000", 2024-09-13T04:59:51.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8279999999999999e-06 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794745+0000", 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0589999999999996e-06 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794748+0000", 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1599999999999998e-06 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837472+0000", 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04272372 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837547+0000", 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4794000000000003e-05 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837620+0000", 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3646999999999995e-05 2024-09-13T04:59:51.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103275 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 414720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.265210874999999, 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042255839000000003, 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103275 2024-09-13T04:59:51.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801780+0000", 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801783+0000", 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801791+0000", 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6830000000000008e-06 2024-09-13T04:59:51.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801799+0000", 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.157e-06 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843846+0000", 2024-09-13T04:59:51.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042047227999999999 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843943+0000", 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7310000000000002e-05 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844037+0000", 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3160999999999998e-05 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.101 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103277 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 428032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.265187379, 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042336314, 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103277 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.102 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801807+0000", 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8090000000000001e-06 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801814+0000", 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6669999999999996e-06 2024-09-13T04:59:51.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801820+0000", 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0419999999999999e-06 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844063+0000", 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042242793000000001 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844103+0000", 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9526000000000001e-05 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844141+0000", 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7892999999999998e-05 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103278 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 441344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:51.106 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.265166110999999, 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042426288, 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103278 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801825+0000", 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801828+0000", 2024-09-13T04:59:51.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.525e-06 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801835+0000", 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8669999999999994e-06 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801841+0000", 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.677e-06 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844165+0000", 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042323652000000003 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844210+0000", 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5623e-05 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844252+0000", 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1384999999999998e-05 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103279 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.265145103, 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042502914000000003, 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103279 2024-09-13T04:59:51.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801846+0000", 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801849+0000", 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8080000000000001e-06 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801858+0000", 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9649999999999997e-06 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801864+0000", 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3119999999999999e-06 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844274+0000", 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042409413999999999 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844313+0000", 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9131999999999997e-05 2024-09-13T04:59:51.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844349+0000", 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6751000000000001e-05 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103280 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 470016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.265122234, 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042583929, 2024-09-13T04:59:51.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103280 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801872+0000", 2024-09-13T04:59:51.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9069999999999999e-06 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801880+0000", 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0800000000000006e-06 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801886+0000", 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2339999999999999e-06 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844372+0000", 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042486084 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844413+0000", 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0448999999999998e-05 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844453+0000", 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550999999999999e-05 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103281 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.265086917, 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042747949, 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103281 2024-09-13T04:59:51.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801904+0000", 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801907+0000", 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1690000000000001e-06 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801920+0000", 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2082000000000001e-05 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801926+0000", 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7360000000000004e-06 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844484+0000", 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042557709999999999 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844552+0000", 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8396e-05 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844653+0000", 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100295 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103282 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 496640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.265057952999999, 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042899236, 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103282 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801933+0000", 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801961+0000", 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7827999999999999e-05 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801971+0000", 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0225e-05 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801981+0000", 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9280000000000005e-06 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844692+0000", 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042711256000000003 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844769+0000", 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6375e-05 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844833+0000", 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4071000000000004e-05 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103283 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.265003862, 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025461000000001, 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103283 2024-09-13T04:59:51.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801987+0000", 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801990+0000", 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2969999999999999e-06 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801999+0000", 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8470000000000002e-06 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802006+0000", 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2169999999999999e-06 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844871+0000", 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042865409 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844950+0000", 2024-09-13T04:59:51.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8670000000000004e-05 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845013+0000", 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-05 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103284 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 523264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:51.133 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.264980673, 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043179033999999998, 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103284 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802010+0000", 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802014+0000", 2024-09-13T04:59:51.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1099999999999999e-06 2024-09-13T04:59:51.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802021+0000", 2024-09-13T04:59:51.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7729999999999999e-06 2024-09-13T04:59:51.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802027+0000", 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845053+0000", 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025270999999997 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845130+0000", 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7387999999999998e-05 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845190+0000", 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9938000000000001e-05 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103531 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 387072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.138 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.256907825000001, 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046484202000000002, 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103531 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810083+0000", 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810087+0000", 2024-09-13T04:59:51.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5090000000000002e-06 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810114+0000", 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7263999999999998e-05 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810122+0000", 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8310000000000001e-06 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851361+0000", 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041238969 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851601+0000", 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024058400000000001 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.856568+0000", 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049666449999999996 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.142 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103537 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.256584948, 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042298980999999999, 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103537 2024-09-13T04:59:51.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810406+0000", 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810409+0000", 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0489999999999999e-06 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810417+0000", 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.763e-06 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810423+0000", 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3779999999999998e-06 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852568+0000", 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042144632000000001 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852644+0000", 2024-09-13T04:59:51.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5908000000000006e-05 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852706+0000", 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1713999999999999e-05 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103538 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 484352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.256544665, 2024-09-13T04:59:51.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042420402000000003, 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103538 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810446+0000", 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810457+0000", 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0686999999999999e-05 2024-09-13T04:59:51.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810486+0000", 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8784000000000001e-05 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810494+0000", 2024-09-13T04:59:51.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5979999999999997e-06 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852727+0000", 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042232984000000001 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852801+0000", 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3691999999999998e-05 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852867+0000", 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6391999999999997e-05 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103539 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.256489480000001, 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042532474000000001, 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103539 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:51.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810501+0000", 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810506+0000", 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1740000000000002e-06 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810515+0000", 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9099999999999994e-06 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810521+0000", 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5359999999999998e-06 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852890+0000", 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042368994 2024-09-13T04:59:51.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852968+0000", 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8097999999999997e-05 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853035+0000", 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6501999999999994e-05 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103540 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 510976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.256464575000001, 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042686832000000001, 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103540 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:51.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810530+0000", 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.88e-06 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810538+0000", 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1729999999999995e-06 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810544+0000", 2024-09-13T04:59:51.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2179999999999996e-06 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853061+0000", 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042516929000000002 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853135+0000", 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4333999999999997e-05 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:51.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853214+0000", 2024-09-13T04:59:51.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8715999999999996e-05 2024-09-13T04:59:51.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.161 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:51.161 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:51.161 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:51.161 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:51.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:51.163 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:51.158+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:51.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T04:59:51.277 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:51.274+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:51.322 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:51.323 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:51.348 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:51.354 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:51.354 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:51.354 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:51.354 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:51.364 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:51.358+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:51.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:51.378 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:51.394 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:51.394 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:51.397 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-13T04:59:51.414 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:51.414 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:51.434 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:51.434 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:51.454 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:51.455 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:51.465 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:51.461+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:51.475 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:51.475 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:51.490 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:51.490 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:51.504 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:51.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-13T04:59:51.566 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:51.561+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:51.654 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:51.655 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:51.667 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:51.661+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:51.668 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:51.768 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:51.761+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:51.799 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:51.799 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:51.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:51.901 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:51.897+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:51.943 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T04:59:51.970 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:51.965+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:51.985 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:51.985 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:52.001 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:52.070 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:52.066+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:52.171 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:52.171 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:52.171 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:52.165+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:52.185 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:52.273 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:52.269+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:52.352 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:52.352 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:52.366 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:52.372 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:52.366+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:52.473 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:52.469+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:52.531 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:52.531 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:52.545 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:52.574 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:52.570+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:52.675 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:52.670+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:52.706 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:52.706 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:52.720 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:52.776 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:52.769+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:52.840 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:52.841 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:52.855 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:52.877 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:52.874+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:52.977 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:52.974+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:53.028 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:53.028 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:53.043 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:53.089 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:53.085+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:53.171 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:53.171 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:53.179 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:53.174+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:53.186 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:53.279 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:53.278+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:53.353 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:53.353 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:53.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:53.413 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:53.409+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:53.481 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:53.478+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:53.483 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:53.483 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:53.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:53.497 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:53.582 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:53.578+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:53.649 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:53.660 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:53.661 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:53.665 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:53.665 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:53.676 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:53.682 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:53.683 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:53.683 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:53.677+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:53.698 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:53.699 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:53.716 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:53.716 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:53.733 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:53.733 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:53.753 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:53.753 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:53.773 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:53.784 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:53.778+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:53.800 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:53.800 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:53.800 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:53.800 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:53.814 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:53.885 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:53.881+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:53.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T04:59:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:53.992 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:53.992 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:53.992 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:53.992 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:53.992 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:53.986+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:54.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:54.097 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:54.094+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:54.176 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.456053690999994, 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053636234999999997, 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.177 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:54.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772488+0000", 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053447254999999999 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772562+0000", 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4276999999999994e-05 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772656+0000", 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3993999999999995e-05 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5898240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.456028298999996, 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054161726, 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:54.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772681+0000", 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053622832000000002 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772751+0000", 2024-09-13T04:59:54.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0166999999999994e-05 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773207+0000", 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045547300000000002 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5931008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.456010848000005, 2024-09-13T04:59:54.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054348847999999998, 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:59:54.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773235+0000", 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054159997000000001 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773316+0000", 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0727000000000001e-05 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773411+0000", 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5266999999999994e-05 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.455993599999999, 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054777692000000003, 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773440+0000", 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054341649999999998 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773514+0000", 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4501999999999999e-05 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773858+0000", 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034316700000000002 2024-09-13T04:59:54.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.196 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.196 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.455970847000003, 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054958912999999998, 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773886+0000", 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054770389000000003 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773964+0000", 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774062+0000", 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7059999999999996e-05 2024-09-13T04:59:54.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.455953897000001, 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055382103000000002, 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.201 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774088+0000", 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054950812000000002 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774163+0000", 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5084000000000001e-05 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774502+0000", 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033872500000000001 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.455932231000006, 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055600325999999999, 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:59:54.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:59:54.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774532+0000", 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055378563999999998 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774633+0000", 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100519 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774742+0000", 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010891900000000001 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6094848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.455915996000002, 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056713126000000003, 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:59:54.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774769+0000", 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055599061999999998 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774843+0000", 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3801999999999996e-05 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775871+0000", 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00102797 2024-09-13T04:59:54.215 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.455899682999998, 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056923247000000003, 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.216 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775941+0000", 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056755141000000002 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775996+0000", 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4580999999999998e-05 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776097+0000", 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101456 2024-09-13T04:59:54.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.221 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.221 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.455883682000007, 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057407979999999997, 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:54.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:54.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:59:54.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:59:54.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:59:54.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:59:54.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:59:54.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:59:54.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:59:54.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776138+0000", 2024-09-13T04:59:54.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056931362999999999 2024-09-13T04:59:54.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776293+0000", 2024-09-13T04:59:54.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015495600000000001 2024-09-13T04:59:54.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776598+0000", 2024-09-13T04:59:54.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030449200000000001 2024-09-13T04:59:54.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.229 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.230 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.230 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.230 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:54.230 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.445294899999993, 2024-09-13T04:59:54.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053642026000000002, 2024-09-13T04:59:54.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783061+0000", 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053263004000000003 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783140+0000", 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9290999999999997e-05 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783421+0000", 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028007100000000001 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.235 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.445270416, 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053832005000000002, 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:59:54.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783450+0000", 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053635404999999997 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783532+0000", 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1310000000000006e-05 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783635+0000", 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010324599999999999 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.445254448, 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05399698, 2024-09-13T04:59:54.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:59:54.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783666+0000", 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053834224999999999 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783745+0000", 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8542999999999997e-05 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783816+0000", 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279999999999995e-05 2024-09-13T04:59:54.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.445221176999993, 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054125706000000003, 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783842+0000", 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053971225999999997 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783915+0000", 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2657999999999995e-05 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783978+0000", 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3157000000000005e-05 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.444057645000001, 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053864002000000001, 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.251 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:59:54.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:59:54.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:59:54.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:59:54.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:59:54.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:59:54.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:59:54.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784436+0000", 2024-09-13T04:59:54.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053408069000000002 2024-09-13T04:59:54.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784643+0000", 2024-09-13T04:59:54.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000207171 2024-09-13T04:59:54.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784880+0000", 2024-09-13T04:59:54.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023680700000000001 2024-09-13T04:59:54.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.258 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.258 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.258 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:54.259 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.444025217000004, 2024-09-13T04:59:54.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054149968, 2024-09-13T04:59:54.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:59:54.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.260 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:54.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:54.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:54.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:59:54.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:59:54.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784958+0000", 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053896985000000001 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785046+0000", 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8321999999999999e-05 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785198+0000", 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015185 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.443363583999997, 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053689033999999997, 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785224+0000", 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053491874000000002 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785304+0000", 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9825000000000002e-05 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785399+0000", 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4837e-05 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.270 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99151 3.f 3:f7f71ef4:::rbd_data.112afaf35b85.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.792988127000001, 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053542113000000002, 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99151 2024-09-13T04:59:54.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382083+0000", 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384133+0000", 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002049871 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384157+0000", 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3620000000000001e-05 2024-09-13T04:59:54.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384170+0000", 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3152999999999999e-05 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384238+0000", 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7883e-05 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384350+0000", 2024-09-13T04:59:54.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112525 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384768+0000", 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041751899999999998 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426232+0000", 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041464366000000002 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435554+0000", 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093217539999999998 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435589+0000", 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4468000000000003e-05 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435627+0000", 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8892e-05 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99154 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.779829531000001, 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058608128000000002, 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.277 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99154 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395242+0000", 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396241+0000", 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00099964600000000008 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396263+0000", 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1939000000000001e-05 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396276+0000", 2024-09-13T04:59:54.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2758999999999999e-05 2024-09-13T04:59:54.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396371+0000", 2024-09-13T04:59:54.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5233000000000001e-05 2024-09-13T04:59:54.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396524+0000", 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015272899999999999 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396949+0000", 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042510200000000001 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.418937+0000", 2024-09-13T04:59:54.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021988085000000001 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453775+0000", 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034838002999999999 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453813+0000", 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8180999999999999e-05 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453852+0000", 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8852e-05 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99155 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.778767637000001, 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057964565000000003, 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.284 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99155 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396305+0000", 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396310+0000", 2024-09-13T04:59:54.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0950000000000003e-06 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396318+0000", 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.191e-06 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396324+0000", 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.959e-06 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416450+0000", 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020125707999999999 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416531+0000", 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1093000000000004e-05 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:54.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416721+0000", 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019040800000000001 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.419351+0000", 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026298229999999999 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454195+0000", 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034844092 2024-09-13T04:59:54.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:54.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454241+0000", 2024-09-13T04:59:54.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5918000000000003e-05 2024-09-13T04:59:54.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454270+0000", 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9391e-05 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.291 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:54.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:54.293 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:54.286+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:54.394 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:54.389+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:54.461 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:54.461 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:54.461 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:54.477 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:54.494 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:54.489+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:54.594 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:54.595 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:54.595 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:54.595 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:54.595 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:54.595 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:54.590+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:54.609 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:54.698 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:54.694+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:54.782 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:54.782 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:54.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:54.782 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:54.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99158 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.381835220999999, 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075195995000000002, 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99158 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397653+0000", 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:54.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400038+0000", 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023859279999999998 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400063+0000", 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4488999999999999e-05 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400077+0000", 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3698e-05 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400154+0000", 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7343999999999997e-05 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400255+0000", 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010082299999999999 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400681+0000", 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042635500000000001 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428967+0000", 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028285346999999999 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472642+0000", 2024-09-13T04:59:54.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043675249999999999 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472724+0000", 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1779000000000004e-05 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472851+0000", 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012780199999999999 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99159 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.379384884, 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.076482954000000006, 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99159 2024-09-13T04:59:54.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400104+0000", 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400110+0000", 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0669999999999997e-06 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400118+0000", 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4120000000000004e-06 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400123+0000", 2024-09-13T04:59:54.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4979999999999997e-06 2024-09-13T04:59:54.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426616+0000", 2024-09-13T04:59:54.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026492418 2024-09-13T04:59:54.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426701+0000", 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4870000000000006e-05 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426882+0000", 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018111000000000001 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429464+0000", 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025825539999999999 2024-09-13T04:59:54.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476516+0000", 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047051760999999998 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476551+0000", 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5317000000000001e-05 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476589+0000", 2024-09-13T04:59:54.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7339999999999998e-05 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106461 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1073152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.633409841999999, 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066606603, 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106461 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:54.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.145628+0000", 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999547 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146105+0000", 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00047782800000000002 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146119+0000", 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193999999999999e-05 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146126+0000", 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4780000000000002e-06 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.206940+0000", 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060814147999999998 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.207024+0000", 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4049999999999997e-05 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.212687+0000", 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005662963 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106751 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1114112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.799 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.618115799000002, 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066917983, 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106751 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161339+0000", 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999647 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161379+0000", 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9650999999999997e-05 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161388+0000", 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0669999999999998e-06 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161395+0000", 2024-09-13T04:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5860000000000001e-06 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.226954+0000", 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065559382999999999 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228142+0000", 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011879659999999999 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228293+0000", 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015084899999999999 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106752 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1140736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.617910803000001, 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067202778000000005, 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106752 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161400+0000", 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998202 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161585+0000", 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018476899999999999 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161604+0000", 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9585e-05 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161612+0000", 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228406+0000", 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066793111000000002 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228594+0000", 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188765 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228783+0000", 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018817500000000001 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.808 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106753 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.617495937000001, 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067030303999999999, 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106753 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161619+0000", 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996243 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162000+0000", 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038113399999999999 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162014+0000", 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4258e-05 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162023+0000", 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2024-09-13T04:59:54.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228821+0000", 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066797788999999996 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228915+0000", 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4655999999999996e-05 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229025+0000", 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010971 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106754 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1167360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.617460496, 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067209986999999999, 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106754 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162029+0000", 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162033+0000", 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1659999999999998e-06 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162058+0000", 2024-09-13T04:59:54.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5944000000000001e-05 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162071+0000", 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2439999999999999e-05 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229059+0000", 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066987645999999998 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229134+0000", 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5780999999999998e-05 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229240+0000", 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105697 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106757 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1127424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:54.817 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.616780399, 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066728500999999996, 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106757 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162079+0000", 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9993687 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162720+0000", 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641229 2024-09-13T04:59:54.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162732+0000", 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1460999999999999e-05 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162739+0000", 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.25e-06 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229267+0000", 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066528114999999999 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229346+0000", 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8710999999999999e-05 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229439+0000", 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2860000000000002e-05 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106765 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1186816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.616745086999998, 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066904627999999994, 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106765 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.822 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:54.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162748+0000", 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.672e-06 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162756+0000", 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8560000000000007e-06 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162792+0000", 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6309000000000001e-05 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229474+0000", 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066682166000000001 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229551+0000", 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7187999999999993e-05 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229650+0000", 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.891e-05 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106767 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1213440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.616471742999998, 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066831682000000003, 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106767 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162799+0000", 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997797 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163024+0000", 2024-09-13T04:59:54.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000225336 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163037+0000", 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2870999999999999e-05 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163045+0000", 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6329999999999996e-06 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229677+0000", 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066632620000000004 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229752+0000", 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4770000000000004e-05 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229851+0000", 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8548999999999994e-05 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106808 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1293312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.613418199000002, 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066071383999999997, 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106808 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166075+0000", 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6450000000000001e-06 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166080+0000", 2024-09-13T04:59:54.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6949999999999997e-06 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166092+0000", 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2102e-05 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.231966+0000", 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065873612999999998 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232047+0000", 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0848999999999998e-05 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232144+0000", 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6971000000000004e-05 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106809 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1306624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.613213577, 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066094452999999997, 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106809 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166099+0000", 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998217 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166282+0000", 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018357900000000001 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166294+0000", 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2074e-05 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166302+0000", 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:54.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232179+0000", 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065877145999999998 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232275+0000", 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5727e-05 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232371+0000", 2024-09-13T04:59:54.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6805999999999994e-05 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106871 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1545216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.609203098999998, 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.071935387000000003, 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106871 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:54.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170290+0000", 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8370000000000001e-06 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170296+0000", 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0009999999999998e-06 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170301+0000", 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9849999999999997e-06 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.234836+0000", 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064535151999999998 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.235101+0000", 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026450200000000001 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.242223+0000", 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0071222799999999999 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107250 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 936960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.584519332999999, 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066117713999999994, 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107250 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194973+0000", 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.973e-06 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194976+0000", 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999998e-06 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194979+0000", 2024-09-13T04:59:54.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9000000000000002e-06 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.259870+0000", 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064890639999999999 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.260247+0000", 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037699199999999998 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261089+0000", 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084223800000000004 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107251 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 950272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.584492280999999, 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066299278000000003, 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107251 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194997+0000", 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195003+0000", 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195013+0000", 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0088999999999999e-05 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195020+0000", 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0859999999999996e-06 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261119+0000", 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066099034000000001 2024-09-13T04:59:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261197+0000", 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7828000000000001e-05 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261298+0000", 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100685 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107264 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.582430799000001, 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066053248999999994, 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107264 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.196135+0000", 2024-09-13T04:59:54.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9990754 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197065+0000", 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093008399999999997 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197076+0000", 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0992000000000001e-05 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197083+0000", 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7059999999999998e-06 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261962+0000", 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064879036000000001 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262040+0000", 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8584999999999994e-05 2024-09-13T04:59:54.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263113+0000", 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010726240000000001 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107265 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.582401475000001, 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264963999999996, 2024-09-13T04:59:54.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107265 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197091+0000", 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197099+0000", 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6459999999999993e-06 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197104+0000", 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0989999999999998e-06 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263144+0000", 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066040233000000004 2024-09-13T04:59:54.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263232+0000", 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7361000000000006e-05 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263354+0000", 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122354 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107266 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1156096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.582381985000001, 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066462531000000005, 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107266 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197108+0000", 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197111+0000", 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.016e-06 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197119+0000", 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660000000000002e-06 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197124+0000", 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2900000000000002e-06 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263389+0000", 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264725999999996 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263464+0000", 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5426000000000006e-05 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263571+0000", 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010699100000000001 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107414 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1699840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.571423770999999, 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069644172000000004, 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107414 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208066+0000", 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208069+0000", 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.396e-06 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208079+0000", 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0569e-05 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208085+0000", 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6339999999999998e-06 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272576+0000", 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064490818000000005 2024-09-13T04:59:54.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272673+0000", 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7427000000000001e-05 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.277711+0000", 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050376939999999997 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107622 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1315840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.558440475000001, 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.073340916000000006, 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107622 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:54.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:54.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221052+0000", 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221059+0000", 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8770000000000004e-06 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221064+0000", 2024-09-13T04:59:54.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284453+0000", 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063388694999999995 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284566+0000", 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113082 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:54.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.294391+0000", 2024-09-13T04:59:54.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00982477 2024-09-13T04:59:54.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:54.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:54.876 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:54.878 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:54.878 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:54.874+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T04:59:54.978 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:54.974+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:55.068 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:55.068 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:55.068 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:55.068 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:55.079 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:55.073+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:55.084 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:55.180 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:55.173+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:55.211 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:55.211 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:55.211 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:55.211 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:55.211 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:55.226 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:55.281 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:55.278+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:55.353 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 98.889809017000005, 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:59:55.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103023 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.556280231999999, 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042297819, 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103023 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794689+0000", 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7060000000000001e-06 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794693+0000", 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.811e-06 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794696+0000", 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1429999999999998e-06 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836839+0000", 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042142577000000001 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836920+0000", 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1012000000000003e-05 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836986+0000", 2024-09-13T04:59:55.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5841999999999997e-05 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103024 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 482304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.556269364999999, 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042470763000000002, 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103024 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794698+0000", 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794700+0000", 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.567e-06 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794704+0000", 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8399999999999997e-06 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794707+0000", 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1209999999999998e-06 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837011+0000", 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042303828000000002 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837096+0000", 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5148999999999994e-05 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837169+0000", 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3521000000000003e-05 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.368 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103025 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.556258533000001, 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042586098000000003, 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103025 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794709+0000", 2024-09-13T04:59:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794711+0000", 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5999999999999999e-06 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794715+0000", 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8260000000000003e-06 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794718+0000", 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837196+0000", 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042477695000000003 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837252+0000", 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6808000000000002e-05 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837296+0000", 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3327e-05 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103026 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 508928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.556247855999999, 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042722826999999998, 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103026 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:55.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794721+0000", 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4619999999999999e-06 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794725+0000", 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7890000000000001e-06 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794736+0000", 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1202999999999999e-05 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837322+0000", 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042585852 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837386+0000", 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3942000000000005e-05 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837443+0000", 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6841e-05 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.378 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103027 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.379 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:55.379 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.556228553, 2024-09-13T04:59:55.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042880907000000003, 2024-09-13T04:59:55.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103027 2024-09-13T04:59:55.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:55.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:55.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794739+0000", 2024-09-13T04:59:55.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794741+0000", 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8279999999999999e-06 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794745+0000", 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0589999999999996e-06 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794748+0000", 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1599999999999998e-06 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837472+0000", 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04272372 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837547+0000", 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4794000000000003e-05 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837620+0000", 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3646999999999995e-05 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103275 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 414720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.549187308, 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042255839000000003, 2024-09-13T04:59:55.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103275 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801780+0000", 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801783+0000", 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-13T04:59:55.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801791+0000", 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6830000000000008e-06 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801799+0000", 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.157e-06 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843846+0000", 2024-09-13T04:59:55.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042047227999999999 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843943+0000", 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7310000000000002e-05 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844037+0000", 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3160999999999998e-05 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.389 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103277 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 428032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.549163812, 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042336314, 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103277 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801807+0000", 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8090000000000001e-06 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801814+0000", 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6669999999999996e-06 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801820+0000", 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0419999999999999e-06 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844063+0000", 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042242793000000001 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844103+0000", 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9526000000000001e-05 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844141+0000", 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7892999999999998e-05 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.394 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103278 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 441344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.549142543999999, 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042426288, 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103278 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801825+0000", 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801828+0000", 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.525e-06 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801835+0000", 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8669999999999994e-06 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801841+0000", 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.677e-06 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844165+0000", 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042323652000000003 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844210+0000", 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5623e-05 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844252+0000", 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1384999999999998e-05 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103279 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.549121536000001, 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042502914000000003, 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103279 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801846+0000", 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801849+0000", 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8080000000000001e-06 2024-09-13T04:59:55.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801858+0000", 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9649999999999997e-06 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801864+0000", 2024-09-13T04:59:55.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3119999999999999e-06 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844274+0000", 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042409413999999999 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844313+0000", 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9131999999999997e-05 2024-09-13T04:59:55.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844349+0000", 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6751000000000001e-05 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103280 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 470016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.549098666999999, 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042583929, 2024-09-13T04:59:55.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103280 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801872+0000", 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9069999999999999e-06 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801880+0000", 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0800000000000006e-06 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801886+0000", 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2339999999999999e-06 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844372+0000", 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042486084 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844413+0000", 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0448999999999998e-05 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844453+0000", 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550999999999999e-05 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.408 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103281 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.549063350000001, 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042747949, 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103281 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801904+0000", 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801907+0000", 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1690000000000001e-06 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801920+0000", 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2082000000000001e-05 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801926+0000", 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7360000000000004e-06 2024-09-13T04:59:55.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844484+0000", 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042557709999999999 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844552+0000", 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8396e-05 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844653+0000", 2024-09-13T04:59:55.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100295 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103282 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 496640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.549034385999999, 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042899236, 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.413 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103282 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801933+0000", 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801961+0000", 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7827999999999999e-05 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801971+0000", 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0225e-05 2024-09-13T04:59:55.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801981+0000", 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9280000000000005e-06 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844692+0000", 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042711256000000003 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844769+0000", 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6375e-05 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844833+0000", 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4071000000000004e-05 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103283 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:55.417 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.548980295, 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025461000000001, 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103283 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801987+0000", 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801990+0000", 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2969999999999999e-06 2024-09-13T04:59:55.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801999+0000", 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8470000000000002e-06 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802006+0000", 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2169999999999999e-06 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844871+0000", 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042865409 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844950+0000", 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8670000000000004e-05 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845013+0000", 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-05 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.421 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103284 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 523264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.548957106, 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043179033999999998, 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103284 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802010+0000", 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802014+0000", 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1099999999999999e-06 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802021+0000", 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7729999999999999e-06 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802027+0000", 2024-09-13T04:59:55.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845053+0000", 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025270999999997 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845130+0000", 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7387999999999998e-05 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845190+0000", 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9938000000000001e-05 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103531 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 387072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.540884257999998, 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046484202000000002, 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103531 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810083+0000", 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810087+0000", 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5090000000000002e-06 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810114+0000", 2024-09-13T04:59:55.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7263999999999998e-05 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810122+0000", 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8310000000000001e-06 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851361+0000", 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041238969 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851601+0000", 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024058400000000001 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.856568+0000", 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049666449999999996 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103537 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:55.430 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.540561381, 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042298980999999999, 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103537 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810406+0000", 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810409+0000", 2024-09-13T04:59:55.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0489999999999999e-06 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810417+0000", 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.763e-06 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810423+0000", 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3779999999999998e-06 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852568+0000", 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042144632000000001 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852644+0000", 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5908000000000006e-05 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852706+0000", 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1713999999999999e-05 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103538 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 484352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.540521097999999, 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042420402000000003, 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103538 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:55.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810446+0000", 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810457+0000", 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0686999999999999e-05 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810486+0000", 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8784000000000001e-05 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810494+0000", 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5979999999999997e-06 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852727+0000", 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042232984000000001 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852801+0000", 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3691999999999998e-05 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852867+0000", 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6391999999999997e-05 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.439 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103539 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.540465912999998, 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042532474000000001, 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103539 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810501+0000", 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810506+0000", 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1740000000000002e-06 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810515+0000", 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9099999999999994e-06 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810521+0000", 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5359999999999998e-06 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852890+0000", 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042368994 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852968+0000", 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8097999999999997e-05 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853035+0000", 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6501999999999994e-05 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103540 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 510976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.540441007999998, 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042686832000000001, 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103540 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810530+0000", 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.88e-06 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810538+0000", 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1729999999999995e-06 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810544+0000", 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2179999999999996e-06 2024-09-13T04:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853061+0000", 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042516929000000002 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853135+0000", 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4333999999999997e-05 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853214+0000", 2024-09-13T04:59:55.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8715999999999996e-05 2024-09-13T04:59:55.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.449 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:55.450 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:55.451 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:55.446+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:55.552 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:55.549+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:55.624 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:55.625 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:55.625 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:55.638 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:55.686 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:55.682+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:55.753 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:55.749+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:55.765 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:55.765 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:55.765 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:55.765 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:55.765 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:55.779 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T04:59:55.793 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T04:59:55.854 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:55.853+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:55.955 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:55.949+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:55.960 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:55.960 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:55.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:55.960 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.79532991000001, 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.961 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T04:59:55.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T04:59:55.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 99.792889251000005, 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T04:59:55.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T04:59:55.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 85.94605731, 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T04:59:55.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.976 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 85.931546257999997, 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:55.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T04:59:55.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:55.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99140 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.369486+0000", 2024-09-13T04:59:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.589822555000001, 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058811783999999999, 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99140 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369483+0000", 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371901+0000", 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024177479999999999 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371935+0000", 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999999e-05 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371949+0000", 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3917000000000001e-05 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372078+0000", 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012926999999999999 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372339+0000", 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000260394 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.373383+0000", 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010445809999999999 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428034+0000", 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054651138000000002 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428093+0000", 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8604000000000003e-05 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428253+0000", 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016013 2024-09-13T04:59:55.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428297+0000", 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4459000000000002e-05 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99141 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.371969+0000", 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.587339123, 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058946757000000002, 2024-09-13T04:59:55.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99141 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371967+0000", 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371974+0000", 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2609999999999999e-06 2024-09-13T04:59:55.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371981+0000", 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4449999999999999e-06 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371986+0000", 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3630000000000002e-06 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424374+0000", 2024-09-13T04:59:55.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.052387356000000003 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424479+0000", 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010508199999999999 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424690+0000", 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021145300000000001 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:55.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429820+0000", 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051295810000000002 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430829+0000", 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010095480000000001 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430878+0000", 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8979e-05 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430916+0000", 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7413000000000002e-05 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106233 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1172480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121778+0000", 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837530560000001, 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054064492999999998, 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:55.994 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106233 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T04:59:55.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121777+0000", 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121793+0000", 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5945999999999999e-05 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121798+0000", 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3030000000000001e-06 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121801+0000", 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.898e-06 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175652+0000", 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053851796 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:55.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:55.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:55.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175743+0000", 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1002999999999996e-05 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175842+0000", 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8745000000000004e-05 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106244 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1191936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121804+0000", 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837504689999999, 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054259089000000003, 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106244 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121803+0000", 2024-09-13T04:59:56.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121805+0000", 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.53e-06 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121808+0000", 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.96e-06 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121810+0000", 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175871+0000", 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054060460999999997 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175956+0000", 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5227000000000002e-05 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176063+0000", 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106764 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106245 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1205248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121812+0000", 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837495910000001, 2024-09-13T04:59:56.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054466459000000002, 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106245 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121814+0000", 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4079999999999999e-06 2024-09-13T04:59:56.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121816+0000", 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8880000000000001e-06 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121819+0000", 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176099+0000", 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054279922000000001 2024-09-13T04:59:56.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176176+0000", 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7495000000000003e-05 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176279+0000", 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102503 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106246 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1218560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121821+0000", 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837487223, 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054660164999999997, 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106246 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:59:56.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121822+0000", 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.342e-06 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121825+0000", 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.005e-06 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121828+0000", 2024-09-13T04:59:56.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.807e-06 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176305+0000", 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054476681999999998 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176387+0000", 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2238999999999996e-05 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176481+0000", 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4287e-05 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106247 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1231872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121830+0000", 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837478113, 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054861114000000002, 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106247 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121831+0000", 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3209999999999999e-06 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121834+0000", 2024-09-13T04:59:56.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121837+0000", 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4339999999999999e-06 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176507+0000", 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054670165 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176591+0000", 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3820999999999996e-05 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176691+0000", 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100739 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106258 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1384448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121839+0000", 2024-09-13T04:59:56.019 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837469410000001, 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055059518000000002, 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106258 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T04:59:56.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121840+0000", 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2890000000000001e-06 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121843+0000", 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8700000000000001e-06 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121845+0000", 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6869999999999999e-06 2024-09-13T04:59:56.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176717+0000", 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054871862 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176801+0000", 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3473999999999994e-05 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176898+0000", 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7535999999999996e-05 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106259 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1397760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121849+0000", 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837459345999999, 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055256324000000002, 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106259 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121851+0000", 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9920000000000002e-06 2024-09-13T04:59:56.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121856+0000", 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5090000000000001e-06 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121861+0000", 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176928+0000", 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055067656999999999 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177010+0000", 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1298000000000007e-05 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177105+0000", 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5623000000000002e-05 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106260 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121863+0000", 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837444879, 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055448655999999999, 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106260 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.029 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:59:56.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121865+0000", 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.553e-06 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121868+0000", 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.106e-06 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121870+0000", 2024-09-13T04:59:56.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.481e-06 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177133+0000", 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055262349000000002 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177212+0000", 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9305e-05 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177312+0000", 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100083 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106261 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1424384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121872+0000", 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837435835000001, 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055627715000000001, 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106261 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121874+0000", 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3379999999999999e-06 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121876+0000", 2024-09-13T04:59:56.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.937e-06 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121879+0000", 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4509999999999999e-06 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177331+0000", 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055451584999999998 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177401+0000", 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0269000000000001e-05 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177500+0000", 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9335999999999999e-05 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106262 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1437696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.037 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121881+0000", 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.837427086000002, 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055839664999999997, 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106262 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T04:59:56.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121883+0000", 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.823e-06 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121888+0000", 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121893+0000", 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.425e-06 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177529+0000", 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05563651 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177616+0000", 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6794999999999998e-05 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177721+0000", 2024-09-13T04:59:56.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104991 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106284 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1245184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123308+0000", 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.836000630000001, 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054629361000000001, 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106284 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T04:59:56.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123307+0000", 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123312+0000", 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9640000000000002e-06 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123324+0000", 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1965e-05 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123332+0000", 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0979999999999994e-06 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177750+0000", 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054418375999999997 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177832+0000", 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1947999999999995e-05 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177937+0000", 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010498299999999999 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.046 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106285 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1258496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123339+0000", 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.835968786999999, 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054821169000000003, 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106285 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123342+0000", 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3519999999999998e-06 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123350+0000", 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6159999999999996e-06 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123355+0000", 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2050000000000001e-06 2024-09-13T04:59:56.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177969+0000", 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054613802000000003 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178053+0000", 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3800000000000004e-05 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178161+0000", 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107876 2024-09-13T04:59:56.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106286 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.130235+0000", 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.829073336, 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054693622999999997, 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106286 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130234+0000", 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130240+0000", 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1519999999999997e-06 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130255+0000", 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4739000000000001e-05 2024-09-13T04:59:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130262+0000", 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5979999999999999e-06 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178190+0000", 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047928040999999998 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178285+0000", 2024-09-13T04:59:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5082999999999997e-05 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.184929+0000", 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066430589999999998 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107657 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1654784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.211654+0000", 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.747653798000002, 2024-09-13T04:59:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054578494999999998, 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107657 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211657+0000", 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1920000000000001e-06 2024-09-13T04:59:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211674+0000", 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6923e-05 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211691+0000", 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6957e-05 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262566+0000", 2024-09-13T04:59:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050875110000000001 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262650+0000", 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.266233+0000", 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035824989999999998 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:56.059 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:56.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T04:59:56.062 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:56.062 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:56.063 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:56.063 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:56.064 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:56.064 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:56.054+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:56.078 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:56.097 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:56.098 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:56.123 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:56.124 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:56.185 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:56.182+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:56.203 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:56.203 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:56.217 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T04:59:56.263 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:56.258+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:56.335 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:56.335 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:56.349 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T04:59:56.397 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:56.393+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:56.465 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:56.461+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:56.469 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:56.469 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:56.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T04:59:56.566 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:56.562+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:56.654 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:56.654 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:56.667 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:56.661+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:56.669 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T04:59:56.767 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T04:59:56.767 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T04:59:56.767 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T04:59:56.767 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T04:59:56.767 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T04:59:56.767 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T04:59:56.768 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:56.761+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:56.835 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:56.835 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:56.849 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T04:59:56.897 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:56.893+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:56.904 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:56.904 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T04:59:56.904 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:56.969 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:56.966+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:56.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T04:59:57.026 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:57.027 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:57.042 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T04:59:57.089 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:57.085+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:57.170 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:57.170 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:57.171 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:57.165+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:57.184 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T04:59:57.272 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:57.269+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:57.344 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:57.345 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:57.358 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T04:59:57.373 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:57.370+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:57.474 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:57.470+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:57.525 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:57.525 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:57.537 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T04:59:57.575 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:57.574+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:57.658 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:57.658 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:57.673 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T04:59:57.676 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:57.670+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:57.777 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:57.770+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:57.796 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:57.797 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:57.811 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T04:59:57.877 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:57.870+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:57.933 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:57.933 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:57.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T04:59:57.978 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:57.973+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T04:59:58.080 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:58.073+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:58.091 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:58.092 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:58.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T04:59:58.143 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T04:59:58.202 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:58.198+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:58.232 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T04:59:58.232 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T04:59:58.248 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T04:59:58.293 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T04:59:58.293 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T04:59:58.294 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:58.290+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:58.310 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T04:59:58.310 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T04:59:58.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T04:59:58.327 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T04:59:58.345 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T04:59:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T04:59:58.366 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T04:59:58.366 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T04:59:58.382 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:58.378+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:58.386 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T04:59:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T04:59:58.406 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T04:59:58.407 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T04:59:58.414 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:58.414 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:58.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:58.414 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:58.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33209 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5865472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692773376000005, 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053636234999999997, 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33209 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719020+0000", 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719019+0000", 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719022+0000", 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.345e-06 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719028+0000", 2024-09-13T04:59:58.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2589999999999997e-06 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719040+0000", 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2471e-05 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772488+0000", 2024-09-13T04:59:58.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053447254999999999 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772562+0000", 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4276999999999994e-05 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772656+0000", 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3993999999999995e-05 2024-09-13T04:59:58.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33210 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5898240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692747983999993, 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054161726, 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33210 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719045+0000", 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719047+0000", 2024-09-13T04:59:58.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5979999999999999e-06 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719054+0000", 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719058+0000", 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6269999999999999e-06 2024-09-13T04:59:58.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772681+0000", 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053622832000000002 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.772751+0000", 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0166999999999994e-05 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773207+0000", 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045547300000000002 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33211 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5931008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692730533000002, 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054348847999999998, 2024-09-13T04:59:58.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33211 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719063+0000", 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719062+0000", 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719064+0000", 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1509999999999998e-06 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719070+0000", 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9529999999999996e-06 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719075+0000", 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0669999999999999e-06 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773235+0000", 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054159997000000001 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773316+0000", 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0727000000000001e-05 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773411+0000", 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5266999999999994e-05 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33212 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692713284999996, 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054777692000000003, 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33212 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719080+0000", 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719079+0000", 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719082+0000", 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4430000000000002e-06 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719088+0000", 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0889999999999996e-06 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719098+0000", 2024-09-13T04:59:58.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0217000000000001e-05 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773440+0000", 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054341649999999998 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773514+0000", 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4501999999999999e-05 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773858+0000", 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034316700000000002 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33213 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692690532, 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054958912999999998, 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33213 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719103+0000", 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719102+0000", 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719104+0000", 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2560000000000001e-06 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719111+0000", 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3010000000000003e-06 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719115+0000", 2024-09-13T04:59:58.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773886+0000", 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054770389000000003 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.773964+0000", 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2024-09-13T04:59:58.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774062+0000", 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7059999999999996e-05 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33214 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:58.440 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692673581999998, 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055382103000000002, 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33214 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.441 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719120+0000", 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719119+0000", 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719126+0000", 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7100000000000001e-06 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719132+0000", 2024-09-13T04:59:58.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4130000000000003e-06 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719137+0000", 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6809999999999997e-06 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774088+0000", 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054950812000000002 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774163+0000", 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5084000000000001e-05 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774502+0000", 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033872500000000001 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.445 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33215 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6062080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692651916000003, 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055600325999999999, 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33215 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719141+0000", 2024-09-13T04:59:58.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719143+0000", 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1880000000000001e-06 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719149+0000", 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9390000000000002e-06 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719154+0000", 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.527e-06 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774532+0000", 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055378563999999998 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774633+0000", 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100519 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774742+0000", 2024-09-13T04:59:58.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010891900000000001 2024-09-13T04:59:58.452 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.452 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.452 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33216 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6094848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692635680999999, 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056713126000000003, 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33216 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719157+0000", 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719159+0000", 2024-09-13T04:59:58.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0820000000000001e-06 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719165+0000", 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.896e-06 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719170+0000", 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6280000000000004e-06 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774769+0000", 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055599061999999998 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.774843+0000", 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3801999999999996e-05 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775871+0000", 2024-09-13T04:59:58.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00102797 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33217 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6127616~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692619367999995, 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056923247000000003, 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.458 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33217 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719174+0000", 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719173+0000", 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719175+0000", 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0470000000000001e-06 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719181+0000", 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8239999999999996e-06 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719186+0000", 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5170000000000001e-06 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775941+0000", 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056755141000000002 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.775996+0000", 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4580999999999998e-05 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776097+0000", 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101456 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33219 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6160384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.463 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.692603367000004, 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057407979999999997, 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33219 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719190+0000", 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719189+0000", 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719191+0000", 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0370000000000001e-06 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719197+0000", 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6760000000000004e-06 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.719207+0000", 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.764e-06 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776138+0000", 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056931362999999999 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776293+0000", 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015495600000000001 2024-09-13T04:59:58.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.776598+0000", 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030449200000000001 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33300 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6045696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.468 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.682014585000005, 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053642026000000002, 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33300 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729779+0000", 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729778+0000", 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729787+0000", 2024-09-13T04:59:58.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5029999999999997e-06 2024-09-13T04:59:58.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729794+0000", 2024-09-13T04:59:58.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7710000000000001e-06 2024-09-13T04:59:58.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729798+0000", 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6820000000000002e-06 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783061+0000", 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053263004000000003 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783140+0000", 2024-09-13T04:59:58.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9290999999999997e-05 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783421+0000", 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028007100000000001 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33301 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.681990100999997, 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053832005000000002, 2024-09-13T04:59:58.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33301 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729803+0000", 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729805+0000", 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6819999999999999e-06 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729811+0000", 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7509999999999997e-06 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729815+0000", 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-06 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783450+0000", 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053635404999999997 2024-09-13T04:59:58.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783532+0000", 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1310000000000006e-05 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783635+0000", 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010324599999999999 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33302 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.681974132999997, 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05399698, 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33302 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729819+0000", 2024-09-13T04:59:58.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729821+0000", 2024-09-13T04:59:58.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1959999999999998e-06 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729827+0000", 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3450000000000002e-06 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729832+0000", 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7090000000000001e-06 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783666+0000", 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053834224999999999 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783745+0000", 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8542999999999997e-05 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783816+0000", 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1279999999999995e-05 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33304 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6144000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.681940862000005, 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054125706000000003, 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33304 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.484 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729852+0000", 2024-09-13T04:59:58.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729854+0000", 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.933e-06 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729860+0000", 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8220000000000003e-06 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.729871+0000", 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1259e-05 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783842+0000", 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053971225999999997 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783915+0000", 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2657999999999995e-05 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.783978+0000", 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3157000000000005e-05 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33309 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6209536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.680777329999998, 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053864002000000001, 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.488 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33309 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731016+0000", 2024-09-13T04:59:58.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731015+0000", 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731018+0000", 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.12e-06 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731024+0000", 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9139999999999996e-06 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731028+0000", 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2529999999999998e-06 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784436+0000", 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053408069000000002 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784643+0000", 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000207171 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784880+0000", 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023680700000000001 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33311 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6242304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:58.492 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.680744902000001, 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054149968, 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33311 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731048+0000", 2024-09-13T04:59:58.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731050+0000", 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0870000000000001e-06 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731056+0000", 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1489999999999999e-06 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731061+0000", 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8790000000000002e-06 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.784958+0000", 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053896985000000001 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785046+0000", 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8321999999999999e-05 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785198+0000", 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015185 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:33313 3.3 3:c64742c0:::rbd_data.112ab86ccaf9.0000000000000000:8 [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:58.497 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.680083268999994, 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053689033999999997, 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 33313 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731710+0000", 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731709+0000", 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731712+0000", 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5459999999999998e-06 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731718+0000", 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-06 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.731732+0000", 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4042e-05 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785224+0000", 2024-09-13T04:59:58.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053491874000000002 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785304+0000", 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9825000000000002e-05 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:23.785399+0000", 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4837e-05 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99151 3.f 3:f7f71ef4:::rbd_data.112afaf35b85.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.029707812000002, 2024-09-13T04:59:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053542113000000002, 2024-09-13T04:59:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99151 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382085+0000", 2024-09-13T04:59:58.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.382083+0000", 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384133+0000", 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002049871 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384157+0000", 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3620000000000001e-05 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384170+0000", 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3152999999999999e-05 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384238+0000", 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7883e-05 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384350+0000", 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112525 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.384768+0000", 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041751899999999998 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426232+0000", 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041464366000000002 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435554+0000", 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093217539999999998 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435589+0000", 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4468000000000003e-05 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.435627+0000", 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8892e-05 2024-09-13T04:59:58.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99154 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.016549216000001, 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058608128000000002, 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99154 2024-09-13T04:59:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:58.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395244+0000", 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.395242+0000", 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999976 2024-09-13T04:59:58.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396241+0000", 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00099964600000000008 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396263+0000", 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1939000000000001e-05 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396276+0000", 2024-09-13T04:59:58.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2758999999999999e-05 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396371+0000", 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5233000000000001e-05 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396524+0000", 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015272899999999999 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:58.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396949+0000", 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042510200000000001 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.418937+0000", 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021988085000000001 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453775+0000", 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034838002999999999 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:58.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453813+0000", 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8180999999999999e-05 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.453852+0000", 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8852e-05 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99155 3.c 3:369d7ae1:::rbd_data.112afaf35b85.0000000000000006:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.015487321999998, 2024-09-13T04:59:58.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.057964565000000003, 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99155 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396306+0000", 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396305+0000", 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396310+0000", 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0950000000000003e-06 2024-09-13T04:59:58.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396318+0000", 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.191e-06 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.396324+0000", 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.959e-06 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416450+0000", 2024-09-13T04:59:58.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020125707999999999 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416531+0000", 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1093000000000004e-05 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 7", 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.416721+0000", 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019040800000000001 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:58.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.419351+0000", 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026298229999999999 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454195+0000", 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034844092 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454241+0000", 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5918000000000003e-05 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:58.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.454270+0000", 2024-09-13T04:59:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9391e-05 2024-09-13T04:59:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:58.520 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:58.520 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:58.521 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T04:59:58.521 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T04:59:58.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T04:59:58.523 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:58.517+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:58.622 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:58.618+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:58.701 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:58.701 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:58.716 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T04:59:58.723 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:58.717+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:58.823 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T04:59:58.818+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T04:59:58.888 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:58.888 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:58.888 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:58.888 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:58.888 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:58.902 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T04:59:58.924 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:58.917+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:59.025 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T04:59:59.018+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T04:59:59.084 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:59.084 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:59.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99158 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.685408185, 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075195995000000002, 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.085 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99158 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397653+0000", 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400038+0000", 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023859279999999998 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400063+0000", 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4488999999999999e-05 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400077+0000", 2024-09-13T04:59:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3698e-05 2024-09-13T04:59:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400154+0000", 2024-09-13T04:59:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7343999999999997e-05 2024-09-13T04:59:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400255+0000", 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010082299999999999 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400681+0000", 2024-09-13T04:59:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042635500000000001 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428967+0000", 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028285346999999999 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472642+0000", 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043675249999999999 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472724+0000", 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1779000000000004e-05 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472851+0000", 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012780199999999999 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99159 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T04:59:59.092 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.682957848000001, 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.076482954000000006, 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99159 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:59.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400104+0000", 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400110+0000", 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0669999999999997e-06 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400118+0000", 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4120000000000004e-06 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400123+0000", 2024-09-13T04:59:59.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4979999999999997e-06 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426616+0000", 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026492418 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426701+0000", 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4870000000000006e-05 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T04:59:59.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426882+0000", 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018111000000000001 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429464+0000", 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025825539999999999 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476516+0000", 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047051760999999998 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476551+0000", 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5317000000000001e-05 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476589+0000", 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7339999999999998e-05 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106461 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1073152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.936982806, 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066606603, 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106461 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.145628+0000", 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999547 2024-09-13T04:59:59.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146105+0000", 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00047782800000000002 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146119+0000", 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193999999999999e-05 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146126+0000", 2024-09-13T04:59:59.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4780000000000002e-06 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.206940+0000", 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060814147999999998 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.207024+0000", 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4049999999999997e-05 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.212687+0000", 2024-09-13T04:59:59.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005662963 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106751 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1114112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.921688762999999, 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066917983, 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106751 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161339+0000", 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999647 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161379+0000", 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9650999999999997e-05 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161388+0000", 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0669999999999998e-06 2024-09-13T04:59:59.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161395+0000", 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5860000000000001e-06 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.226954+0000", 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065559382999999999 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228142+0000", 2024-09-13T04:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011879659999999999 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228293+0000", 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015084899999999999 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106752 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1140736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:59.107 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.921483767000002, 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067202778000000005, 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106752 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161400+0000", 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998202 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161585+0000", 2024-09-13T04:59:59.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018476899999999999 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161604+0000", 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9585e-05 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161612+0000", 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228406+0000", 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066793111000000002 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228594+0000", 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188765 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228783+0000", 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018817500000000001 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106753 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.921068901000002, 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067030303999999999, 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106753 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161619+0000", 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996243 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162000+0000", 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038113399999999999 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162014+0000", 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4258e-05 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162023+0000", 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228821+0000", 2024-09-13T04:59:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066797788999999996 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228915+0000", 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4655999999999996e-05 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229025+0000", 2024-09-13T04:59:59.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010971 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106754 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1167360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.92103346, 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067209986999999999, 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.117 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106754 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162029+0000", 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162033+0000", 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1659999999999998e-06 2024-09-13T04:59:59.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162058+0000", 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5944000000000001e-05 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162071+0000", 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2439999999999999e-05 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229059+0000", 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066987645999999998 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229134+0000", 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5780999999999998e-05 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229240+0000", 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105697 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106757 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1127424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.920353363, 2024-09-13T04:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066728500999999996, 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106757 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162079+0000", 2024-09-13T04:59:59.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9993687 2024-09-13T04:59:59.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162720+0000", 2024-09-13T04:59:59.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641229 2024-09-13T04:59:59.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162732+0000", 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1460999999999999e-05 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162739+0000", 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.25e-06 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229267+0000", 2024-09-13T04:59:59.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066528114999999999 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229346+0000", 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8710999999999999e-05 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229439+0000", 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2860000000000002e-05 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106765 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1186816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.920318050999999, 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066904627999999994, 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106765 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162748+0000", 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.672e-06 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162756+0000", 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8560000000000007e-06 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162792+0000", 2024-09-13T04:59:59.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6309000000000001e-05 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229474+0000", 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066682166000000001 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229551+0000", 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7187999999999993e-05 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229650+0000", 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.891e-05 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106767 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1213440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.920044706999999, 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066831682000000003, 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.132 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106767 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T04:59:59.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162799+0000", 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997797 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163024+0000", 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000225336 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163037+0000", 2024-09-13T04:59:59.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2870999999999999e-05 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163045+0000", 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6329999999999996e-06 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229677+0000", 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066632620000000004 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229752+0000", 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4770000000000004e-05 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229851+0000", 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8548999999999994e-05 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106808 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1293312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.916991162999999, 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066071383999999997, 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106808 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166075+0000", 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6450000000000001e-06 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166080+0000", 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6949999999999997e-06 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166092+0000", 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2102e-05 2024-09-13T04:59:59.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.231966+0000", 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065873612999999998 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232047+0000", 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0848999999999998e-05 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232144+0000", 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6971000000000004e-05 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106809 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1306624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.916786541, 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066094452999999997, 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.141 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106809 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166099+0000", 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998217 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166282+0000", 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018357900000000001 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166294+0000", 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2074e-05 2024-09-13T04:59:59.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166302+0000", 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5480000000000004e-06 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232179+0000", 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065877145999999998 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232275+0000", 2024-09-13T04:59:59.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5727e-05 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232371+0000", 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6805999999999994e-05 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106871 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1545216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.912776062999999, 2024-09-13T04:59:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.071935387000000003, 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106871 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170290+0000", 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8370000000000001e-06 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170296+0000", 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0009999999999998e-06 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170301+0000", 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9849999999999997e-06 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.234836+0000", 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064535151999999998 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.235101+0000", 2024-09-13T04:59:59.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026450200000000001 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.242223+0000", 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0071222799999999999 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107250 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 936960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.150 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.888092297, 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066117713999999994, 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107250 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194973+0000", 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.973e-06 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194976+0000", 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999998e-06 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194979+0000", 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9000000000000002e-06 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.259870+0000", 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064890639999999999 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.260247+0000", 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037699199999999998 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261089+0000", 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084223800000000004 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107251 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 950272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:59.156 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.888065245, 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066299278000000003, 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107251 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.157 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194997+0000", 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195003+0000", 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195013+0000", 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0088999999999999e-05 2024-09-13T04:59:59.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195020+0000", 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0859999999999996e-06 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261119+0000", 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066099034000000001 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261197+0000", 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7828000000000001e-05 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261298+0000", 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100685 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107264 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:59.161 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.886003763000001, 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066053248999999994, 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107264 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.162 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.196135+0000", 2024-09-13T04:59:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9990754 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197065+0000", 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093008399999999997 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197076+0000", 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0992000000000001e-05 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197083+0000", 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7059999999999998e-06 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261962+0000", 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064879036000000001 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262040+0000", 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8584999999999994e-05 2024-09-13T04:59:59.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263113+0000", 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010726240000000001 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107265 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.885974439000002, 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264963999999996, 2024-09-13T04:59:59.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107265 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197091+0000", 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197099+0000", 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6459999999999993e-06 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197104+0000", 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0989999999999998e-06 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263144+0000", 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066040233000000004 2024-09-13T04:59:59.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263232+0000", 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7361000000000006e-05 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263354+0000", 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122354 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.170 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107266 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1156096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.885954948999998, 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066462531000000005, 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107266 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:59.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197108+0000", 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197111+0000", 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.016e-06 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197119+0000", 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660000000000002e-06 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197124+0000", 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2900000000000002e-06 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263389+0000", 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264725999999996 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263464+0000", 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5426000000000006e-05 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263571+0000", 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010699100000000001 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107414 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1699840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.874996735, 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069644172000000004, 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107414 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.175 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208066+0000", 2024-09-13T04:59:59.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208069+0000", 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.396e-06 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208079+0000", 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0569e-05 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208085+0000", 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6339999999999998e-06 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272576+0000", 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064490818000000005 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272673+0000", 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7427000000000001e-05 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.277711+0000", 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050376939999999997 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107622 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1315840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.862013439000002, 2024-09-13T04:59:59.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.073340916000000006, 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107622 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221052+0000", 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221059+0000", 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8770000000000004e-06 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221064+0000", 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4639999999999997e-06 2024-09-13T04:59:59.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284453+0000", 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063388694999999995 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284566+0000", 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113082 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.294391+0000", 2024-09-13T04:59:59.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00982477 2024-09-13T04:59:59.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.184 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:59.185 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T04:59:59.186 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T04:59:59.182+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T04:59:59.287 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:59.281+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T04:59:59.361 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:59.361 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:59.362 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:59.362 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:59.373 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T04:59:59.388 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:59.385+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:59.489 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T04:59:59.482+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T04:59:59.500 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:59.500 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:59.500 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T04:59:59.500 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T04:59:59.500 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:59.514 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T04:59:59.590 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T04:59:59.585+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 103.221324029, 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T04:59:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T04:59:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T04:59:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103023 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.887795243999999, 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042297819, 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103023 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794689+0000", 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7060000000000001e-06 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794693+0000", 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.811e-06 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794696+0000", 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1429999999999998e-06 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836839+0000", 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042142577000000001 2024-09-13T04:59:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836920+0000", 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1012000000000003e-05 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836986+0000", 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5841999999999997e-05 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103024 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 482304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.887784376999999, 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042470763000000002, 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103024 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794698+0000", 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794700+0000", 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.567e-06 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794704+0000", 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8399999999999997e-06 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794707+0000", 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1209999999999998e-06 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837011+0000", 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042303828000000002 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837096+0000", 2024-09-13T04:59:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5148999999999994e-05 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837169+0000", 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3521000000000003e-05 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103025 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.887773545000002, 2024-09-13T04:59:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042586098000000003, 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103025 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794709+0000", 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794711+0000", 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5999999999999999e-06 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794715+0000", 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8260000000000003e-06 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794718+0000", 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1089999999999998e-06 2024-09-13T04:59:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837196+0000", 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042477695000000003 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837252+0000", 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6808000000000002e-05 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837296+0000", 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3327e-05 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103026 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 508928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.887762867999999, 2024-09-13T04:59:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042722826999999998, 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103026 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794721+0000", 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4619999999999999e-06 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794725+0000", 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7890000000000001e-06 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794736+0000", 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1202999999999999e-05 2024-09-13T04:59:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837322+0000", 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042585852 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837386+0000", 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3942000000000005e-05 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837443+0000", 2024-09-13T04:59:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6841e-05 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103027 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.887743565000001, 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042880907000000003, 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103027 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T04:59:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794739+0000", 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794741+0000", 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8279999999999999e-06 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794745+0000", 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0589999999999996e-06 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794748+0000", 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1599999999999998e-06 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837472+0000", 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04272372 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837547+0000", 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4794000000000003e-05 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837620+0000", 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3646999999999995e-05 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103275 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 414720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880702320000001, 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042255839000000003, 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103275 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801780+0000", 2024-09-13T04:59:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801783+0000", 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801791+0000", 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6830000000000008e-06 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801799+0000", 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.157e-06 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843846+0000", 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042047227999999999 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843943+0000", 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7310000000000002e-05 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844037+0000", 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3160999999999998e-05 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103277 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 428032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880678824, 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042336314, 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103277 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801807+0000", 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8090000000000001e-06 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801814+0000", 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6669999999999996e-06 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801820+0000", 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0419999999999999e-06 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844063+0000", 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042242793000000001 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844103+0000", 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9526000000000001e-05 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844141+0000", 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7892999999999998e-05 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103278 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 441344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880657555999999, 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042426288, 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103278 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801825+0000", 2024-09-13T04:59:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801828+0000", 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.525e-06 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801835+0000", 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8669999999999994e-06 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801841+0000", 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.677e-06 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844165+0000", 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042323652000000003 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844210+0000", 2024-09-13T04:59:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5623e-05 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844252+0000", 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1384999999999998e-05 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103279 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880636547999998, 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042502914000000003, 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103279 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801846+0000", 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801849+0000", 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8080000000000001e-06 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801858+0000", 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9649999999999997e-06 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801864+0000", 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3119999999999999e-06 2024-09-13T04:59:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844274+0000", 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042409413999999999 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844313+0000", 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9131999999999997e-05 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844349+0000", 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6751000000000001e-05 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103280 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 470016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880613679, 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042583929, 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103280 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801872+0000", 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9069999999999999e-06 2024-09-13T04:59:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801880+0000", 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0800000000000006e-06 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801886+0000", 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2339999999999999e-06 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844372+0000", 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042486084 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844413+0000", 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0448999999999998e-05 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844453+0000", 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550999999999999e-05 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103281 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880578362000001, 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042747949, 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103281 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801904+0000", 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801907+0000", 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1690000000000001e-06 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801920+0000", 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2082000000000001e-05 2024-09-13T04:59:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801926+0000", 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7360000000000004e-06 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844484+0000", 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042557709999999999 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844552+0000", 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8396e-05 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844653+0000", 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100295 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103282 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 496640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880549397999999, 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042899236, 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103282 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801933+0000", 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801961+0000", 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7827999999999999e-05 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801971+0000", 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0225e-05 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801981+0000", 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9280000000000005e-06 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844692+0000", 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042711256000000003 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844769+0000", 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6375e-05 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844833+0000", 2024-09-13T04:59:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4071000000000004e-05 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103283 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880495307, 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025461000000001, 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103283 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T04:59:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801987+0000", 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801990+0000", 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2969999999999999e-06 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801999+0000", 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8470000000000002e-06 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802006+0000", 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2169999999999999e-06 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844871+0000", 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042865409 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844950+0000", 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8670000000000004e-05 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845013+0000", 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-05 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103284 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 523264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.880472118, 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043179033999999998, 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103284 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802010+0000", 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802014+0000", 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1099999999999999e-06 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802021+0000", 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7729999999999999e-06 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802027+0000", 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-13T04:59:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845053+0000", 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025270999999997 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845130+0000", 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7387999999999998e-05 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845190+0000", 2024-09-13T04:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9938000000000001e-05 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103531 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 387072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.872399269999999, 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046484202000000002, 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103531 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810083+0000", 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810087+0000", 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5090000000000002e-06 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810114+0000", 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7263999999999998e-05 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810122+0000", 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8310000000000001e-06 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851361+0000", 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041238969 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851601+0000", 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024058400000000001 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.856568+0000", 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049666449999999996 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103537 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.872076393, 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042298980999999999, 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103537 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810406+0000", 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810409+0000", 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0489999999999999e-06 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810417+0000", 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.763e-06 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810423+0000", 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3779999999999998e-06 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852568+0000", 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042144632000000001 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852644+0000", 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5908000000000006e-05 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852706+0000", 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1713999999999999e-05 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103538 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 484352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.87203611, 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042420402000000003, 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103538 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810446+0000", 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810457+0000", 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0686999999999999e-05 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810486+0000", 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8784000000000001e-05 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810494+0000", 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5979999999999997e-06 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852727+0000", 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042232984000000001 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852801+0000", 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3691999999999998e-05 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852867+0000", 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6391999999999997e-05 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103539 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.871980924999999, 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042532474000000001, 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103539 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810501+0000", 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T04:59:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810506+0000", 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1740000000000002e-06 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810515+0000", 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9099999999999994e-06 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810521+0000", 2024-09-13T04:59:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5359999999999998e-06 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852890+0000", 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042368994 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852968+0000", 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8097999999999997e-05 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853035+0000", 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6501999999999994e-05 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103540 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 510976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.871956019999999, 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042686832000000001, 2024-09-13T04:59:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103540 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T04:59:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810530+0000", 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.88e-06 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810538+0000", 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1729999999999995e-06 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810544+0000", 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2179999999999996e-06 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T04:59:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853061+0000", 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042516929000000002 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853135+0000", 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4333999999999997e-05 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853214+0000", 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8715999999999996e-05 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.777 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.777 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T04:59:59.777 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T04:59:59.777 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:59.778 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T04:59:59.779 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T04:59:59.773+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T04:59:59.879 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T04:59:59.873+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T04:59:59.907 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T04:59:59.908 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T04:59:59.908 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T04:59:59.908 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T04:59:59.922 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T04:59:59.980 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T04:59:59.973+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:00.012 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:00:00.063 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:00.063 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:00.063 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T05:00:00.064 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T05:00:00.064 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:00.081 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:00:00.081 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:00.074+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:00.182 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:00.178+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 126.1096605, 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.277 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T05:00:00.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 104.107219841, 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T05:00:00.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.286 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.260387899999998, 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T05:00:00.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T05:00:00.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.245876847999995, 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:00.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T05:00:00.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T05:00:00.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99140 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.904153144999999, 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058811783999999999, 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99140 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369483+0000", 2024-09-13T05:00:00.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371901+0000", 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024177479999999999 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371935+0000", 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999999e-05 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371949+0000", 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3917000000000001e-05 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372078+0000", 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012926999999999999 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372339+0000", 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000260394 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.373383+0000", 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010445809999999999 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428034+0000", 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054651138000000002 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428093+0000", 2024-09-13T05:00:00.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8604000000000003e-05 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428253+0000", 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016013 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428297+0000", 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4459000000000002e-05 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99141 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.901669713, 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058946757000000002, 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99141 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371967+0000", 2024-09-13T05:00:00.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371974+0000", 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2609999999999999e-06 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371981+0000", 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371986+0000", 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3630000000000002e-06 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424374+0000", 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.052387356000000003 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424479+0000", 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010508199999999999 2024-09-13T05:00:00.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424690+0000", 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021145300000000001 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429820+0000", 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051295810000000002 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430829+0000", 2024-09-13T05:00:00.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010095480000000001 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430878+0000", 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8979e-05 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430916+0000", 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7413000000000002e-05 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106233 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1172480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151861149999998, 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054064492999999998, 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106233 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:00.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121777+0000", 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121793+0000", 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5945999999999999e-05 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121798+0000", 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3030000000000001e-06 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121801+0000", 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.898e-06 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175652+0000", 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053851796 2024-09-13T05:00:00.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175743+0000", 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1002999999999996e-05 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175842+0000", 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8745000000000004e-05 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106244 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1191936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.15183528, 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054259089000000003, 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106244 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121803+0000", 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121805+0000", 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.53e-06 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121808+0000", 2024-09-13T05:00:00.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.96e-06 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121810+0000", 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175871+0000", 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054060460999999997 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175956+0000", 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5227000000000002e-05 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176063+0000", 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106764 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106245 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1205248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.321 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151826499999999, 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054466459000000002, 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106245 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:00.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121814+0000", 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4079999999999999e-06 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121816+0000", 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8880000000000001e-06 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121819+0000", 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176099+0000", 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054279922000000001 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176176+0000", 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7495000000000003e-05 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176279+0000", 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102503 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.325 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106246 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1218560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151817813000001, 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054660164999999997, 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106246 2024-09-13T05:00:00.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121822+0000", 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.342e-06 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121825+0000", 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.005e-06 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121828+0000", 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.807e-06 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176305+0000", 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054476681999999998 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176387+0000", 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2238999999999996e-05 2024-09-13T05:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176481+0000", 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4287e-05 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106247 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1231872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151808703, 2024-09-13T05:00:00.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054861114000000002, 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106247 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121831+0000", 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3209999999999999e-06 2024-09-13T05:00:00.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121834+0000", 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121837+0000", 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176507+0000", 2024-09-13T05:00:00.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054670165 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176591+0000", 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3820999999999996e-05 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176691+0000", 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100739 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106258 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1384448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151800000000001, 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055059518000000002, 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106258 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121840+0000", 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2890000000000001e-06 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121843+0000", 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8700000000000001e-06 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121845+0000", 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6869999999999999e-06 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176717+0000", 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054871862 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176801+0000", 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3473999999999994e-05 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176898+0000", 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7535999999999996e-05 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106259 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1397760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151789936, 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055256324000000002, 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106259 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:00.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121851+0000", 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9920000000000002e-06 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121856+0000", 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5090000000000001e-06 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121861+0000", 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176928+0000", 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055067656999999999 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177010+0000", 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1298000000000007e-05 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177105+0000", 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5623000000000002e-05 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106260 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:00.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151775469, 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055448655999999999, 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106260 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121865+0000", 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.553e-06 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121868+0000", 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.106e-06 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121870+0000", 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.481e-06 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177133+0000", 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055262349000000002 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177212+0000", 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9305e-05 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177312+0000", 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100083 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.347 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106261 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1424384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151766425000002, 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055627715000000001, 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106261 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121874+0000", 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3379999999999999e-06 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121876+0000", 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.937e-06 2024-09-13T05:00:00.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121879+0000", 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4509999999999999e-06 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177331+0000", 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055451584999999998 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177401+0000", 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0269000000000001e-05 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177500+0000", 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9335999999999999e-05 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106262 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1437696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.151757675999999, 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055839664999999997, 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106262 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.353 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121883+0000", 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.823e-06 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121888+0000", 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121893+0000", 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.425e-06 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177529+0000", 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05563651 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177616+0000", 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6794999999999998e-05 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177721+0000", 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104991 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106284 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1245184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.150331219999998, 2024-09-13T05:00:00.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054629361000000001, 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106284 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123307+0000", 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123312+0000", 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9640000000000002e-06 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123324+0000", 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1965e-05 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123332+0000", 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0979999999999994e-06 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177750+0000", 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054418375999999997 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177832+0000", 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1947999999999995e-05 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177937+0000", 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010498299999999999 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.361 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106285 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1258496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.150299377, 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054821169000000003, 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106285 2024-09-13T05:00:00.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:00.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123342+0000", 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3519999999999998e-06 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123350+0000", 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6159999999999996e-06 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123355+0000", 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2050000000000001e-06 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177969+0000", 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054613802000000003 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178053+0000", 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3800000000000004e-05 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178161+0000", 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107876 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.366 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106286 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.143403926000001, 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054693622999999997, 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106286 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130234+0000", 2024-09-13T05:00:00.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130240+0000", 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1519999999999997e-06 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130255+0000", 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4739000000000001e-05 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130262+0000", 2024-09-13T05:00:00.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5979999999999999e-06 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178190+0000", 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047928040999999998 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178285+0000", 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5082999999999997e-05 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.184929+0000", 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066430589999999998 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107657 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1654784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.061984387999999, 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054578494999999998, 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:00.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107657 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211657+0000", 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1920000000000001e-06 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211674+0000", 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6923e-05 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211691+0000", 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6957e-05 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262566+0000", 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050875110000000001 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262650+0000", 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-13T05:00:00.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.266233+0000", 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035824989999999998 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:00.375 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:00.376 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:00:00.381 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:00.373+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:00.477 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:00.474+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:00.514 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:00.514 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:00.514 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:00.514 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:00.523 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:00:00.528 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:00:00.578 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:00.574+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:00.645 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:00.645 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:00.645 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:00.645 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:00.646 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:00.659 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:00:00.679 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:00.674+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:00.683 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:00.683 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:00.700 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:00.700 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:00.716 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:00.716 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:00.732 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:00.733 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:00.753 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:00.754 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:00.776 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:00.777 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:00.780 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:00.773+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:00.783 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:00.783 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:00.795 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:00.795 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:00.796 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:00:00.817 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:00.817 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:00.882 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:00.878+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:00.962 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:00.962 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:00.976 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:00:00.982 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:00.978+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:00.992 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T05:00:01.082 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:01.078+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:01.110 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:01.110 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:01.131 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:00:01.184 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:01.178+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:01.252 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:01.253 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:01.267 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:00:01.285 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:01.278+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:01.386 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:01.382+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:01.388 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:01.389 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:01.403 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:00:01.486 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:01.482+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:01.520 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:01.520 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:01.534 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:00:01.587 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:01.581+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:01.651 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:01.651 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:01.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:00:01.709 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:01.705+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:01.790 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:01.785+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:01.830 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:01.830 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:01.844 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:00:01.891 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:01.886+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:01.920 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:00:01.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T05:00:01.992 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:01.985+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:02.014 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:02.014 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:02.031 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:00:02.121 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:02.117+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:02.193 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:02.189+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:02.206 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:02.206 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:02.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:00:02.294 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:02.290+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:02.345 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:02.345 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:02.359 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:00:02.395 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:02.389+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:02.496 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:02.493+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:02.525 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:02.525 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:02.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:00:02.597 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:02.593+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:02.657 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:02.657 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:02.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:00:02.718 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:02.714+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:02.799 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:02.799 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:02.799 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:02.794+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:02.813 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:00:02.838 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:00:02.901 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:02.898+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:02.980 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:02.980 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:02.996 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:00:03.000 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:03.001 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:03.001 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:02.997+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:03.015 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T05:00:03.022 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:03.022 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:03.023 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:00:03.041 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:03.041 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:03.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T05:00:03.062 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:03.062 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:03.084 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:03.085 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:03.101 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:03.097+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:03.105 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:03.129 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:03.141 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99158 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.397655+0000", 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.742123112000002, 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075195995000000002, 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99158 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397655+0000", 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.397653+0000", 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400038+0000", 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023859279999999998 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400063+0000", 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4488999999999999e-05 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400077+0000", 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3698e-05 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400154+0000", 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7343999999999997e-05 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400255+0000", 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010082299999999999 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400681+0000", 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042635500000000001 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428967+0000", 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028285346999999999 2024-09-13T05:00:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472642+0000", 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043675249999999999 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472724+0000", 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1779000000000004e-05 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.472851+0000", 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012780199999999999 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99159 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.400106+0000", 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.739672774999999, 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.076482954000000006, 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99159 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400106+0000", 2024-09-13T05:00:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400104+0000", 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400110+0000", 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0669999999999997e-06 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400118+0000", 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4120000000000004e-06 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.400123+0000", 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4979999999999997e-06 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426616+0000", 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026492418 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426701+0000", 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4870000000000006e-05 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 5", 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.426882+0000", 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018111000000000001 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429464+0000", 2024-09-13T05:00:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025825539999999999 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476516+0000", 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047051760999999998 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476551+0000", 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5317000000000001e-05 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.476589+0000", 2024-09-13T05:00:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7339999999999998e-05 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106461 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1073152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.146081+0000", 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.993697733000001, 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066606603, 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106461 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146081+0000", 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.145628+0000", 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999547 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146105+0000", 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00047782800000000002 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146119+0000", 2024-09-13T05:00:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193999999999999e-05 2024-09-13T05:00:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.146126+0000", 2024-09-13T05:00:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4780000000000002e-06 2024-09-13T05:00:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.206940+0000", 2024-09-13T05:00:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.060814147999999998 2024-09-13T05:00:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.207024+0000", 2024-09-13T05:00:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4049999999999997e-05 2024-09-13T05:00:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.212687+0000", 2024-09-13T05:00:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005662963 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106751 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1114112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161375+0000", 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.97840369, 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066917983, 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106751 2024-09-13T05:00:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161375+0000", 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161339+0000", 2024-09-13T05:00:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999647 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161379+0000", 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9650999999999997e-05 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161388+0000", 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0669999999999998e-06 2024-09-13T05:00:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161395+0000", 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5860000000000001e-06 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.226954+0000", 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065559382999999999 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228142+0000", 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011879659999999999 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228293+0000", 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015084899999999999 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106752 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1140736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161580+0000", 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.978198694, 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067202778000000005, 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106752 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161580+0000", 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161400+0000", 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998202 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161585+0000", 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018476899999999999 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161604+0000", 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9585e-05 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161612+0000", 2024-09-13T05:00:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.106e-06 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228406+0000", 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066793111000000002 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228594+0000", 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188765 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228783+0000", 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018817500000000001 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106753 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.161995+0000", 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.977783828, 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067030303999999999, 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106753 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161995+0000", 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.161619+0000", 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9996243 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162000+0000", 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00038113399999999999 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162014+0000", 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4258e-05 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162023+0000", 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228821+0000", 2024-09-13T05:00:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066797788999999996 2024-09-13T05:00:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.228915+0000", 2024-09-13T05:00:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4655999999999996e-05 2024-09-13T05:00:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229025+0000", 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010971 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106754 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1167360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162030+0000", 2024-09-13T05:00:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.977748386999998, 2024-09-13T05:00:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067209986999999999, 2024-09-13T05:00:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106754 2024-09-13T05:00:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162030+0000", 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162029+0000", 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162033+0000", 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1659999999999998e-06 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162058+0000", 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5944000000000001e-05 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162071+0000", 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2439999999999999e-05 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229059+0000", 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066987645999999998 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229134+0000", 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5780999999999998e-05 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229240+0000", 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105697 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106757 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1127424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162710+0000", 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.977068289999998, 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066728500999999996, 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106757 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162710+0000", 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162079+0000", 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9993687 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162720+0000", 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641229 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162732+0000", 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1460999999999999e-05 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162739+0000", 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.25e-06 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229267+0000", 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066528114999999999 2024-09-13T05:00:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229346+0000", 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8710999999999999e-05 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229439+0000", 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2860000000000002e-05 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106765 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1186816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.162745+0000", 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.977032978, 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066904627999999994, 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106765 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162745+0000", 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162748+0000", 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.672e-06 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162756+0000", 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8560000000000007e-06 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162792+0000", 2024-09-13T05:00:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6309000000000001e-05 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229474+0000", 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066682166000000001 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229551+0000", 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7187999999999993e-05 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229650+0000", 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.891e-05 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106767 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1213440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.163019+0000", 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.976759634, 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066831682000000003, 2024-09-13T05:00:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106767 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163019+0000", 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.162799+0000", 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9997797 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163024+0000", 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000225336 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163037+0000", 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2870999999999999e-05 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.163045+0000", 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6329999999999996e-06 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229677+0000", 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066632620000000004 2024-09-13T05:00:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229752+0000", 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4770000000000004e-05 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.229851+0000", 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8548999999999994e-05 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106808 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1293312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166072+0000", 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.97370609, 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066071383999999997, 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106808 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166072+0000", 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166075+0000", 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6450000000000001e-06 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166080+0000", 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6949999999999997e-06 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166092+0000", 2024-09-13T05:00:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2102e-05 2024-09-13T05:00:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.231966+0000", 2024-09-13T05:00:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065873612999999998 2024-09-13T05:00:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232047+0000", 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0848999999999998e-05 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232144+0000", 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6971000000000004e-05 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106809 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1306624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.166277+0000", 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.973501467999998, 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066094452999999997, 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106809 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166277+0000", 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166099+0000", 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9998217 2024-09-13T05:00:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166282+0000", 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018357900000000001 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166294+0000", 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2074e-05 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.166302+0000", 2024-09-13T05:00:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5480000000000004e-06 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232179+0000", 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.065877145999999998 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232275+0000", 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5727e-05 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.232371+0000", 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6805999999999994e-05 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106871 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1545216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.170287+0000", 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.969490990000001, 2024-09-13T05:00:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.071935387000000003, 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106871 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170287+0000", 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170290+0000", 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170296+0000", 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0009999999999998e-06 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.170301+0000", 2024-09-13T05:00:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9849999999999997e-06 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.234836+0000", 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064535151999999998 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.235101+0000", 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026450200000000001 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.242223+0000", 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0071222799999999999 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107250 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 936960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194971+0000", 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.944807224000002, 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066117713999999994, 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107250 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194971+0000", 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194973+0000", 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.973e-06 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194976+0000", 2024-09-13T05:00:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999998e-06 2024-09-13T05:00:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194979+0000", 2024-09-13T05:00:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9000000000000002e-06 2024-09-13T05:00:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.259870+0000", 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064890639999999999 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.260247+0000", 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037699199999999998 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261089+0000", 2024-09-13T05:00:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084223800000000004 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107251 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 950272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.194998+0000", 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.944780172000002, 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066299278000000003, 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107251 2024-09-13T05:00:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194998+0000", 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.194997+0000", 2024-09-13T05:00:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195003+0000", 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5099999999999998e-06 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195013+0000", 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0088999999999999e-05 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.195020+0000", 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0859999999999996e-06 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261119+0000", 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066099034000000001 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261197+0000", 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7828000000000001e-05 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261298+0000", 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100685 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107264 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197060+0000", 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.94271869, 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066053248999999994, 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107264 2024-09-13T05:00:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T05:00:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197060+0000", 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.196135+0000", 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9990754 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197065+0000", 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093008399999999997 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197076+0000", 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0992000000000001e-05 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197083+0000", 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7059999999999998e-06 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.261962+0000", 2024-09-13T05:00:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064879036000000001 2024-09-13T05:00:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262040+0000", 2024-09-13T05:00:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8584999999999994e-05 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263113+0000", 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010726240000000001 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107265 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197089+0000", 2024-09-13T05:00:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.942689366, 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264963999999996, 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107265 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T05:00:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197089+0000", 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197091+0000", 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.79e-06 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197099+0000", 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6459999999999993e-06 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197104+0000", 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0989999999999998e-06 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263144+0000", 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066040233000000004 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263232+0000", 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7361000000000006e-05 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263354+0000", 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122354 2024-09-13T05:00:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107266 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1156096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.197109+0000", 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.942669876, 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066462531000000005, 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107266 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197109+0000", 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197108+0000", 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197111+0000", 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.016e-06 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197119+0000", 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660000000000002e-06 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.197124+0000", 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2900000000000002e-06 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263389+0000", 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.066264725999999996 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263464+0000", 2024-09-13T05:00:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5426000000000006e-05 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.263571+0000", 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010699100000000001 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107414 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1699840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.208067+0000", 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.931711662000001, 2024-09-13T05:00:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069644172000000004, 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107414 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T05:00:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208067+0000", 2024-09-13T05:00:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208066+0000", 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208069+0000", 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.396e-06 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208079+0000", 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0569e-05 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.208085+0000", 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6339999999999998e-06 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272576+0000", 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064490818000000005 2024-09-13T05:00:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.272673+0000", 2024-09-13T05:00:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7427000000000001e-05 2024-09-13T05:00:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.277711+0000", 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050376939999999997 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107622 3.10 3:0e6e15fc:::rbd_data.112afaf35b85.0000000000000003:29 [read 1315840~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.221050+0000", 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 26.918728366, 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.073340916000000006, 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107622 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T05:00:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221050+0000", 2024-09-13T05:00:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221052+0000", 2024-09-13T05:00:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3350000000000001e-06 2024-09-13T05:00:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221059+0000", 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8770000000000004e-06 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.221064+0000", 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4639999999999997e-06 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284453+0000", 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.063388694999999995 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.284566+0000", 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113082 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.294391+0000", 2024-09-13T05:00:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00982477 2024-09-13T05:00:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.250 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:03.251 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:03.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:00:03.253 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:03.246+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:03.353 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:03.346+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:03.403 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:03.403 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:03.403 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T05:00:03.403 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:03.415 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:00:03.454 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:03.449+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:03.554 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:03.549+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:03.580 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:03.580 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:03.581 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T05:00:03.581 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T05:00:03.581 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:03.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:00:03.656 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:03.650+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.244714495, 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T05:00:03.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:03.710 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:03.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:03.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T05:00:03.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:03.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T05:00:03.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T05:00:03.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T05:00:03.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T05:00:03.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T05:00:03.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T05:00:03.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T05:00:03.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T05:00:03.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T05:00:03.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T05:00:03.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:03.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T05:00:03.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T05:00:03.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T05:00:03.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T05:00:03.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.718 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.719 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103023 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.719 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:03.719 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.911185710000002, 2024-09-13T05:00:03.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042297819, 2024-09-13T05:00:03.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.720 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103023 2024-09-13T05:00:03.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.721 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:03.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:03.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:03.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794689+0000", 2024-09-13T05:00:03.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7060000000000001e-06 2024-09-13T05:00:03.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794693+0000", 2024-09-13T05:00:03.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.811e-06 2024-09-13T05:00:03.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794696+0000", 2024-09-13T05:00:03.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1429999999999998e-06 2024-09-13T05:00:03.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836839+0000", 2024-09-13T05:00:03.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042142577000000001 2024-09-13T05:00:03.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836920+0000", 2024-09-13T05:00:03.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1012000000000003e-05 2024-09-13T05:00:03.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836986+0000", 2024-09-13T05:00:03.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5841999999999997e-05 2024-09-13T05:00:03.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.729 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.729 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103024 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 482304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.730 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:03.730 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.911174843000001, 2024-09-13T05:00:03.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042470763000000002, 2024-09-13T05:00:03.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103024 2024-09-13T05:00:03.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.731 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:03.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:03.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794698+0000", 2024-09-13T05:00:03.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794700+0000", 2024-09-13T05:00:03.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.567e-06 2024-09-13T05:00:03.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794704+0000", 2024-09-13T05:00:03.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8399999999999997e-06 2024-09-13T05:00:03.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794707+0000", 2024-09-13T05:00:03.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1209999999999998e-06 2024-09-13T05:00:03.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837011+0000", 2024-09-13T05:00:03.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042303828000000002 2024-09-13T05:00:03.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837096+0000", 2024-09-13T05:00:03.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5148999999999994e-05 2024-09-13T05:00:03.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837169+0000", 2024-09-13T05:00:03.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3521000000000003e-05 2024-09-13T05:00:03.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.738 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.739 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.739 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103025 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.739 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:03.739 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.911164011, 2024-09-13T05:00:03.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042586098000000003, 2024-09-13T05:00:03.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103025 2024-09-13T05:00:03.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.741 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:03.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794709+0000", 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794711+0000", 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5999999999999999e-06 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794715+0000", 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8260000000000003e-06 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794718+0000", 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837196+0000", 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042477695000000003 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837252+0000", 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6808000000000002e-05 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837296+0000", 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3327e-05 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103026 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 508928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.911153334000002, 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042722826999999998, 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103026 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794721+0000", 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4619999999999999e-06 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794725+0000", 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7890000000000001e-06 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794736+0000", 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1202999999999999e-05 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837322+0000", 2024-09-13T05:00:03.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042585852 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837386+0000", 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3942000000000005e-05 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837443+0000", 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6841e-05 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103027 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.911134031, 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042880907000000003, 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103027 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.750 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794739+0000", 2024-09-13T05:00:03.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794741+0000", 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8279999999999999e-06 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794745+0000", 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0589999999999996e-06 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794748+0000", 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837472+0000", 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04272372 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837547+0000", 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4794000000000003e-05 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837620+0000", 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3646999999999995e-05 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103275 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 414720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.904092786, 2024-09-13T05:00:03.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042255839000000003, 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103275 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801780+0000", 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801783+0000", 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-13T05:00:03.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801791+0000", 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6830000000000008e-06 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801799+0000", 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.157e-06 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843846+0000", 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042047227999999999 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843943+0000", 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7310000000000002e-05 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844037+0000", 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3160999999999998e-05 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103277 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 428032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.904069289999999, 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042336314, 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103277 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.759 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:03.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801807+0000", 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8090000000000001e-06 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801814+0000", 2024-09-13T05:00:03.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6669999999999996e-06 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801820+0000", 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0419999999999999e-06 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844063+0000", 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042242793000000001 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844103+0000", 2024-09-13T05:00:03.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9526000000000001e-05 2024-09-13T05:00:03.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844141+0000", 2024-09-13T05:00:03.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7892999999999998e-05 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103278 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 441344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.904048022000001, 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042426288, 2024-09-13T05:00:03.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103278 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801825+0000", 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801828+0000", 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.525e-06 2024-09-13T05:00:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801835+0000", 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8669999999999994e-06 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801841+0000", 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.677e-06 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844165+0000", 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042323652000000003 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844210+0000", 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5623e-05 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844252+0000", 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1384999999999998e-05 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103279 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.904027014, 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042502914000000003, 2024-09-13T05:00:03.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103279 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801846+0000", 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801849+0000", 2024-09-13T05:00:03.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801858+0000", 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9649999999999997e-06 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801864+0000", 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3119999999999999e-06 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844274+0000", 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042409413999999999 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844313+0000", 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9131999999999997e-05 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844349+0000", 2024-09-13T05:00:03.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6751000000000001e-05 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103280 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 470016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.904004145000002, 2024-09-13T05:00:03.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042583929, 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103280 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801872+0000", 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801880+0000", 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0800000000000006e-06 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801886+0000", 2024-09-13T05:00:03.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2339999999999999e-06 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844372+0000", 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042486084 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844413+0000", 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0448999999999998e-05 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844453+0000", 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550999999999999e-05 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103281 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.903968828, 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042747949, 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103281 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801904+0000", 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801907+0000", 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1690000000000001e-06 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801920+0000", 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2082000000000001e-05 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801926+0000", 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7360000000000004e-06 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844484+0000", 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042557709999999999 2024-09-13T05:00:03.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844552+0000", 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8396e-05 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844653+0000", 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100295 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103282 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 496640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.903939864000002, 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042899236, 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103282 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801933+0000", 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801961+0000", 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7827999999999999e-05 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801971+0000", 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0225e-05 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801981+0000", 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9280000000000005e-06 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844692+0000", 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042711256000000003 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844769+0000", 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6375e-05 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844833+0000", 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4071000000000004e-05 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103283 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.903885772999999, 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025461000000001, 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103283 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801987+0000", 2024-09-13T05:00:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801990+0000", 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2969999999999999e-06 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801999+0000", 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8470000000000002e-06 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802006+0000", 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2169999999999999e-06 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844871+0000", 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042865409 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844950+0000", 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8670000000000004e-05 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845013+0000", 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-05 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103284 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 523264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.903862583999999, 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043179033999999998, 2024-09-13T05:00:03.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103284 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802010+0000", 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802014+0000", 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1099999999999999e-06 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802021+0000", 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7729999999999999e-06 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802027+0000", 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-13T05:00:03.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845053+0000", 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025270999999997 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845130+0000", 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7387999999999998e-05 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845190+0000", 2024-09-13T05:00:03.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9938000000000001e-05 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103531 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 387072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.895789736000001, 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046484202000000002, 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103531 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810083+0000", 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810087+0000", 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5090000000000002e-06 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810114+0000", 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7263999999999998e-05 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810122+0000", 2024-09-13T05:00:03.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8310000000000001e-06 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851361+0000", 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041238969 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851601+0000", 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024058400000000001 2024-09-13T05:00:03.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.856568+0000", 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049666449999999996 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103537 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.895466858999999, 2024-09-13T05:00:03.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042298980999999999, 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103537 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810406+0000", 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810409+0000", 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0489999999999999e-06 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810417+0000", 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.763e-06 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810423+0000", 2024-09-13T05:00:03.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3779999999999998e-06 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852568+0000", 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042144632000000001 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852644+0000", 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5908000000000006e-05 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852706+0000", 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1713999999999999e-05 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103538 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 484352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.895426575999998, 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042420402000000003, 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.808 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103538 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810446+0000", 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810457+0000", 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0686999999999999e-05 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810486+0000", 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8784000000000001e-05 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810494+0000", 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5979999999999997e-06 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852727+0000", 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042232984000000001 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852801+0000", 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3691999999999998e-05 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852867+0000", 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6391999999999997e-05 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103539 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.895371391000001, 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042532474000000001, 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103539 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810501+0000", 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:03.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810506+0000", 2024-09-13T05:00:03.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1740000000000002e-06 2024-09-13T05:00:03.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810515+0000", 2024-09-13T05:00:03.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9099999999999994e-06 2024-09-13T05:00:03.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810521+0000", 2024-09-13T05:00:03.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5359999999999998e-06 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852890+0000", 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042368994 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852968+0000", 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8097999999999997e-05 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853035+0000", 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6501999999999994e-05 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103540 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 510976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 27.895346486000001, 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042686832000000001, 2024-09-13T05:00:03.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103540 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:03.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:03.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:03.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:03.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:03.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:03.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:03.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810530+0000", 2024-09-13T05:00:03.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.88e-06 2024-09-13T05:00:03.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:03.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810538+0000", 2024-09-13T05:00:03.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1729999999999995e-06 2024-09-13T05:00:03.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:03.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810544+0000", 2024-09-13T05:00:03.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2179999999999996e-06 2024-09-13T05:00:03.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:03.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853061+0000", 2024-09-13T05:00:03.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042516929000000002 2024-09-13T05:00:03.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:03.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853135+0000", 2024-09-13T05:00:03.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4333999999999997e-05 2024-09-13T05:00:03.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:03.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:03.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:03.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853214+0000", 2024-09-13T05:00:03.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8715999999999996e-05 2024-09-13T05:00:03.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.825 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:03.826 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:03.826 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:03.828 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:00:03.829 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:03.826+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:03.929 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:03.922+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T05:00:03.932 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T05:00:03.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T05:00:04.023 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:04.023 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:04.023 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T05:00:04.023 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:04.029 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:04.025+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:04.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:00:04.133 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:04.125+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:04.197 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:04.197 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:04.197 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T05:00:04.197 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T05:00:04.197 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:04.210 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:00:04.257 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:04.254+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:04.322 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:04.322 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:04.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:04.322 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 130.15606011400001, 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.323 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T05:00:04.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 108.153619455, 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.306787514000007, 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T05:00:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T05:00:04.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T05:00:04.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:04.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.292276462000004, 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.338 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T05:00:04.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T05:00:04.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:04.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99140 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.950552759000001, 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058811783999999999, 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99140 2024-09-13T05:00:04.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369483+0000", 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371901+0000", 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024177479999999999 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371935+0000", 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999999e-05 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371949+0000", 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3917000000000001e-05 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372078+0000", 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012926999999999999 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372339+0000", 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000260394 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.373383+0000", 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010445809999999999 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428034+0000", 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054651138000000002 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428093+0000", 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8604000000000003e-05 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428253+0000", 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016013 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428297+0000", 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4459000000000002e-05 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.349 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99141 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.948069326999999, 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058946757000000002, 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99141 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371967+0000", 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:04.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371974+0000", 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2609999999999999e-06 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371981+0000", 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371986+0000", 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3630000000000002e-06 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424374+0000", 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.052387356000000003 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424479+0000", 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010508199999999999 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424690+0000", 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021145300000000001 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429820+0000", 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051295810000000002 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430829+0000", 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010095480000000001 2024-09-13T05:00:04.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430878+0000", 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8979e-05 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430916+0000", 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7413000000000002e-05 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106233 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1172480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198260764, 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054064492999999998, 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106233 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121777+0000", 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:04.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121793+0000", 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5945999999999999e-05 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121798+0000", 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3030000000000001e-06 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121801+0000", 2024-09-13T05:00:04.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.898e-06 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175652+0000", 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053851796 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175743+0000", 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1002999999999996e-05 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175842+0000", 2024-09-13T05:00:04.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8745000000000004e-05 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106244 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1191936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198234893999999, 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054259089000000003, 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106244 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121803+0000", 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121805+0000", 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.53e-06 2024-09-13T05:00:04.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121808+0000", 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.96e-06 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121810+0000", 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175871+0000", 2024-09-13T05:00:04.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054060460999999997 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175956+0000", 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5227000000000002e-05 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176063+0000", 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106764 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106245 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1205248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198226114000001, 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054466459000000002, 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106245 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.365 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121814+0000", 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4079999999999999e-06 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121816+0000", 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8880000000000001e-06 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121819+0000", 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176099+0000", 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054279922000000001 2024-09-13T05:00:04.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176176+0000", 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7495000000000003e-05 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176279+0000", 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102503 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106246 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1218560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198217426999999, 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054660164999999997, 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106246 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:04.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121822+0000", 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.342e-06 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121825+0000", 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.005e-06 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121828+0000", 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.807e-06 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176305+0000", 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054476681999999998 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176387+0000", 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2238999999999996e-05 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176481+0000", 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4287e-05 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106247 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1231872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198208316999999, 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054861114000000002, 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106247 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:04.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121831+0000", 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3209999999999999e-06 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121834+0000", 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121837+0000", 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176507+0000", 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054670165 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176591+0000", 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3820999999999996e-05 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176691+0000", 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100739 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.378 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106258 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1384448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198199614, 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055059518000000002, 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106258 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121840+0000", 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2890000000000001e-06 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121843+0000", 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8700000000000001e-06 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121845+0000", 2024-09-13T05:00:04.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6869999999999999e-06 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176717+0000", 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054871862 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176801+0000", 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3473999999999994e-05 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176898+0000", 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7535999999999996e-05 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106259 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1397760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198189549999999, 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055256324000000002, 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.383 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106259 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:04.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121851+0000", 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9920000000000002e-06 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121856+0000", 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5090000000000001e-06 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121861+0000", 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176928+0000", 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055067656999999999 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177010+0000", 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1298000000000007e-05 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177105+0000", 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5623000000000002e-05 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106260 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.387 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198175082999999, 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055448655999999999, 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106260 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121865+0000", 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.553e-06 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121868+0000", 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.106e-06 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121870+0000", 2024-09-13T05:00:04.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.481e-06 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177133+0000", 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055262349000000002 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177212+0000", 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9305e-05 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177312+0000", 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100083 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106261 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1424384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198166039, 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055627715000000001, 2024-09-13T05:00:04.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106261 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121874+0000", 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3379999999999999e-06 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121876+0000", 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.937e-06 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121879+0000", 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4509999999999999e-06 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177331+0000", 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055451584999999998 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177401+0000", 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0269000000000001e-05 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177500+0000", 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9335999999999999e-05 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106262 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1437696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.198157290000001, 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055839664999999997, 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106262 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121883+0000", 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.823e-06 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121888+0000", 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121893+0000", 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.425e-06 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177529+0000", 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05563651 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177616+0000", 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6794999999999998e-05 2024-09-13T05:00:04.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177721+0000", 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104991 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106284 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1245184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.196730834, 2024-09-13T05:00:04.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054629361000000001, 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106284 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123307+0000", 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123312+0000", 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9640000000000002e-06 2024-09-13T05:00:04.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123324+0000", 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1965e-05 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123332+0000", 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0979999999999994e-06 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177750+0000", 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054418375999999997 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177832+0000", 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1947999999999995e-05 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177937+0000", 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010498299999999999 2024-09-13T05:00:04.405 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106285 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1258496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.196698991000002, 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054821169000000003, 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.406 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106285 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123342+0000", 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3519999999999998e-06 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123350+0000", 2024-09-13T05:00:04.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6159999999999996e-06 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123355+0000", 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2050000000000001e-06 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177969+0000", 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054613802000000003 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178053+0000", 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3800000000000004e-05 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178161+0000", 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107876 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106286 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:04.410 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.18980354, 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054693622999999997, 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106286 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130234+0000", 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130240+0000", 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1519999999999997e-06 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130255+0000", 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4739000000000001e-05 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130262+0000", 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5979999999999999e-06 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178190+0000", 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047928040999999998 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178285+0000", 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5082999999999997e-05 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.184929+0000", 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066430589999999998 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107657 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1654784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.108384002000001, 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054578494999999998, 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107657 2024-09-13T05:00:04.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211657+0000", 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1920000000000001e-06 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211674+0000", 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6923e-05 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211691+0000", 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6957e-05 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262566+0000", 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050875110000000001 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262650+0000", 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-13T05:00:04.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.266233+0000", 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035824989999999998 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:04.419 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:04.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:00:04.422 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:04.417+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:04.522 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:04.522+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:04.553 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:04.553 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:04.553 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:04.553 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:04.568 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:00:04.623 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:04.617+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:04.692 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:04.693 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:04.693 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:04.693 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:04.693 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:04.708 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:00:04.749 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:04.745+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:04.826 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:04.821+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:04.873 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:04.874 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.012520831000003, 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.876 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:04.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T05:00:04.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.880 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.012493919999997, 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.008748284000006, 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.885 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:04.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.889 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.005204958999997, 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T05:00:04.893 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.005165331000001, 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.894 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T05:00:04.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.005141671000004, 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T05:00:04.898 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:04.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T05:00:04.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.005054122000004, 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T05:00:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.005001465999996, 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:04.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.911 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.001075575000002, 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.912 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.913 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T05:00:04.914 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T05:00:04.915 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.001049949999995, 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.916 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.917 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T05:00:04.918 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.919 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:04.920 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 71.001016555999996, 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:04.921 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.922 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.923 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T05:00:04.924 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.996226626999999, 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.925 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.926 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.927 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T05:00:04.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.992831998, 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T05:00:04.929 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T05:00:04.932 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.992759321999998, 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.934 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:04.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.936 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.937 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.989016961000004, 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.938 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.939 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T05:00:04.940 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.941 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.988843970000005, 2024-09-13T05:00:04.942 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T05:00:04.944 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.945 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.946 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.985001370999996, 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.947 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T05:00:04.949 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.950 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:04.951 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.945153392999998, 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T05:00:04.953 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.954 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.955 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.940824171000003, 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T05:00:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T05:00:04.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.959 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 70.933095171000005, 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:04.960 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:04.961 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.962 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.963 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:04.964 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:04.966 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:00:04.966 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:04.962+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:05.026 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T05:00:05.067 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T05:00:05.067 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:05.062+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:05.157 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:05.157 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:05.157 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:05.157 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:05.168 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:05.162+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:05.173 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:00:05.254 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:00:05.268 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:05.261+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:05.289 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:05.289 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:05.289 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:05.289 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:05.289 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:05.304 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:00:05.389 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:05.385+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:05.405 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:05.405 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:05.421 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:05.422 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:05.438 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:05.438 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:05.455 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:05.455 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:05.468 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:05.469 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:05.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:05.469 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:05.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.469 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:05.469 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:05.469 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.43182300799999, 2024-09-13T05:00:05.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:05.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T05:00:05.471 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T05:00:05.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.431777371, 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.475 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:05.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T05:00:05.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T05:00:05.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.479 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.431729287, 2024-09-13T05:00:05.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T05:00:05.481 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.481 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.481 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.481 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.481 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.481 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T05:00:05.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T05:00:05.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.485 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:05.486 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:05.486 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.431633205, 2024-09-13T05:00:05.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T05:00:05.486 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.486 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.486 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.486 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.488 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T05:00:05.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.488 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T05:00:05.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:05.492 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.431581834, 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T05:00:05.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T05:00:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.43153578, 2024-09-13T05:00:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T05:00:05.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T05:00:05.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.43148571899999, 2024-09-13T05:00:05.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T05:00:05.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 109.31407052900001, 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.508 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:05.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T05:00:05.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T05:00:05.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.222617801, 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T05:00:05.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T05:00:05.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.519 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.222536448, 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.22246691799999, 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T05:00:05.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T05:00:05.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T05:00:05.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T05:00:05.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T05:00:05.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T05:00:05.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.222439642, 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.530 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T05:00:05.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.22234119700001, 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.536 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:05.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T05:00:05.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.22227162999999, 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T05:00:05.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T05:00:05.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.545 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.222201156, 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.546 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:05.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T05:00:05.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.222175471, 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.551 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T05:00:05.553 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.555 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.222140283, 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.556 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T05:00:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T05:00:05.558 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T05:00:05.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.559 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T05:00:05.559 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T05:00:05.559 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.559 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.559 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.560 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T05:00:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T05:00:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T05:00:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T05:00:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.22211576700001, 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.564 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:05.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.565 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.565 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T05:00:05.565 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:05.565 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.565 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T05:00:05.566 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T05:00:05.567 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T05:00:05.567 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.567 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.567 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.567 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.568 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.569 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T05:00:05.569 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T05:00:05.569 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.569 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.569 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.569 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.569 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.569 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:05.570 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:05.570 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 104.22204643000001, 2024-09-13T05:00:05.570 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T05:00:05.570 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.570 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:05.570 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.570 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.571 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:05.572 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.572 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.572 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.572 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.572 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.573 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T05:00:05.574 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.575 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:99156 3.0 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 33.069260174, 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083885007999999997, 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 99156 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.576 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396500+0000", 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999933 2024-09-13T05:00:05.577 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404915+0000", 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084155649999999999 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404964+0000", 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8795000000000003e-05 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404987+0000", 2024-09-13T05:00:05.578 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2719999999999999e-05 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405217+0000", 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230163 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405435+0000", 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000218054 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.579 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 7", 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.406287+0000", 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00085163700000000003 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.470407+0000", 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.064120636999999994 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480294+0000", 2024-09-13T05:00:05.580 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098868949999999997 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480345+0000", 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1116999999999999e-05 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:05.581 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480391+0000", 2024-09-13T05:00:05.582 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6224999999999999e-05 2024-09-13T05:00:05.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.582 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:05.582 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:05.582 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:05.583 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:05.583 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:05.584 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:05.585 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:00:05.585 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:05.577+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:05.585 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:05.586 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:05.586 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:05.586 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:05.684 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:05.677+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:05.755 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:05.755 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:05.769 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:00:05.785 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:05.782+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:05.886 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:05.881+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:05.932 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:05.933 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:05.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:00:05.969 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:00:05.989 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:05.985+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:06.026 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T05:00:06.055 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T05:00:06.087 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:06.081+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:06.137 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:00:06.188 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:06.182+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:06.289 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:06.285+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:06.310 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:06.310 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:06.325 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:00:06.417 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:06.413+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:06.490 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:06.486+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:06.494 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:06.494 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:06.508 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:00:06.591 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:06.586+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:06.682 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:06.682 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:06.692 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:06.685+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:06.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:00:06.793 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:06.789+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:06.833 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:06.834 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:06.848 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:00:06.894 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:06.890+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:06.967 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:06.967 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:06.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:00:07.026 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:07.022+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:07.042 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T05:00:07.042 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T05:00:07.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T05:00:07.096 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:07.089+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:07.155 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:07.155 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:07.172 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:00:07.197 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:07.193+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:07.298 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:07.294+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:07.338 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:07.338 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:07.351 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:00:07.399 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:07.393+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:07.500 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:07.493+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:07.518 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:07.518 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:07.533 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:00:07.587 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:00:07.633 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:07.630+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:07.657 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:07.658 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:07.673 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:00:07.718 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:07.714+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:07.725 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:07.725 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:07.741 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:07.756 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:07.757 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:07.774 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:07.774 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:07.794 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:07.801 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:07.797+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:07.814 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:07.814 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:07.830 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:07.830 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:07.830 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T05:00:07.831 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:07.834 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:07.835 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:07.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:00:07.855 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:07.855 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:07.917 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:07.914+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:07.983 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:07.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:07.983 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T05:00:07.984 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T05:00:07.984 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:07.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:00:08.046 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:08.042+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:08.048 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T05:00:08.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T05:00:08.104 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:08.101+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:08.194 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:08.194 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:08.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.73172869699999, 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T05:00:08.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T05:00:08.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103023 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.398199912000003, 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042297819, 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.200 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103023 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:08.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794689+0000", 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7060000000000001e-06 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794693+0000", 2024-09-13T05:00:08.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.811e-06 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794696+0000", 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1429999999999998e-06 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836839+0000", 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042142577000000001 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836920+0000", 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1012000000000003e-05 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836986+0000", 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5841999999999997e-05 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.205 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103024 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 482304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.206 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:08.206 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.398189045000002, 2024-09-13T05:00:08.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042470763000000002, 2024-09-13T05:00:08.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.206 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103024 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:08.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794698+0000", 2024-09-13T05:00:08.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794700+0000", 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.567e-06 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794704+0000", 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8399999999999997e-06 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794707+0000", 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1209999999999998e-06 2024-09-13T05:00:08.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837011+0000", 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042303828000000002 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837096+0000", 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5148999999999994e-05 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837169+0000", 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3521000000000003e-05 2024-09-13T05:00:08.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103025 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.398178213000001, 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042586098000000003, 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103025 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794709+0000", 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794711+0000", 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5999999999999999e-06 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794715+0000", 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8260000000000003e-06 2024-09-13T05:00:08.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794718+0000", 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837196+0000", 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042477695000000003 2024-09-13T05:00:08.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837252+0000", 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6808000000000002e-05 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837296+0000", 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3327e-05 2024-09-13T05:00:08.215 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103026 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 508928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.398167536000003, 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042722826999999998, 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.216 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103026 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:08.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:08.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794721+0000", 2024-09-13T05:00:08.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4619999999999999e-06 2024-09-13T05:00:08.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794725+0000", 2024-09-13T05:00:08.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7890000000000001e-06 2024-09-13T05:00:08.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794736+0000", 2024-09-13T05:00:08.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1202999999999999e-05 2024-09-13T05:00:08.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837322+0000", 2024-09-13T05:00:08.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042585852 2024-09-13T05:00:08.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837386+0000", 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3942000000000005e-05 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837443+0000", 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6841e-05 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103027 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.398148233000001, 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042880907000000003, 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103027 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794739+0000", 2024-09-13T05:00:08.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794741+0000", 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8279999999999999e-06 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794745+0000", 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0589999999999996e-06 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794748+0000", 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837472+0000", 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04272372 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837547+0000", 2024-09-13T05:00:08.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4794000000000003e-05 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837620+0000", 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3646999999999995e-05 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103275 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 414720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.391106987999997, 2024-09-13T05:00:08.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042255839000000003, 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103275 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801780+0000", 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801783+0000", 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801791+0000", 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6830000000000008e-06 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801799+0000", 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.157e-06 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843846+0000", 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042047227999999999 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843943+0000", 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7310000000000002e-05 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844037+0000", 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3160999999999998e-05 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103277 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 428032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.391083492, 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042336314, 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.233 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103277 2024-09-13T05:00:08.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.233 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801807+0000", 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8090000000000001e-06 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801814+0000", 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6669999999999996e-06 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801820+0000", 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0419999999999999e-06 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844063+0000", 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042242793000000001 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844103+0000", 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9526000000000001e-05 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844141+0000", 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7892999999999998e-05 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103278 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 441344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.237 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.391062224000002, 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042426288, 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103278 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:08.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:08.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801825+0000", 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801828+0000", 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.525e-06 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801835+0000", 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8669999999999994e-06 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801841+0000", 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.677e-06 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844165+0000", 2024-09-13T05:00:08.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042323652000000003 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844210+0000", 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5623e-05 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844252+0000", 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1384999999999998e-05 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103279 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.391041215999998, 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042502914000000003, 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103279 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801846+0000", 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801849+0000", 2024-09-13T05:00:08.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801858+0000", 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9649999999999997e-06 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801864+0000", 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3119999999999999e-06 2024-09-13T05:00:08.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844274+0000", 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042409413999999999 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844313+0000", 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9131999999999997e-05 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844349+0000", 2024-09-13T05:00:08.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6751000000000001e-05 2024-09-13T05:00:08.248 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.248 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.248 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103280 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 470016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.391018346999999, 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042583929, 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103280 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801872+0000", 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801880+0000", 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0800000000000006e-06 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801886+0000", 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2339999999999999e-06 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844372+0000", 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042486084 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844413+0000", 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0448999999999998e-05 2024-09-13T05:00:08.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844453+0000", 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550999999999999e-05 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103281 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.253 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.390983030000001, 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042747949, 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103281 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801904+0000", 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801907+0000", 2024-09-13T05:00:08.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1690000000000001e-06 2024-09-13T05:00:08.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801920+0000", 2024-09-13T05:00:08.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2082000000000001e-05 2024-09-13T05:00:08.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801926+0000", 2024-09-13T05:00:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7360000000000004e-06 2024-09-13T05:00:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844484+0000", 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042557709999999999 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844552+0000", 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8396e-05 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844653+0000", 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100295 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.258 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103282 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 496640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.390954065999999, 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042899236, 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103282 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801933+0000", 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801961+0000", 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7827999999999999e-05 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801971+0000", 2024-09-13T05:00:08.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0225e-05 2024-09-13T05:00:08.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801981+0000", 2024-09-13T05:00:08.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9280000000000005e-06 2024-09-13T05:00:08.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844692+0000", 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042711256000000003 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844769+0000", 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6375e-05 2024-09-13T05:00:08.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844833+0000", 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4071000000000004e-05 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103283 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:08.265 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.390899975000004, 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025461000000001, 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103283 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801987+0000", 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801990+0000", 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2969999999999999e-06 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801999+0000", 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8470000000000002e-06 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802006+0000", 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2169999999999999e-06 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844871+0000", 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042865409 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844950+0000", 2024-09-13T05:00:08.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8670000000000004e-05 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845013+0000", 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-05 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103284 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 523264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:08.270 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.390876786, 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043179033999999998, 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103284 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802010+0000", 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802014+0000", 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1099999999999999e-06 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802021+0000", 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7729999999999999e-06 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802027+0000", 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845053+0000", 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025270999999997 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845130+0000", 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7387999999999998e-05 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845190+0000", 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9938000000000001e-05 2024-09-13T05:00:08.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103531 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 387072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.382803938000002, 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046484202000000002, 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103531 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810083+0000", 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810087+0000", 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5090000000000002e-06 2024-09-13T05:00:08.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810114+0000", 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7263999999999998e-05 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810122+0000", 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8310000000000001e-06 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851361+0000", 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041238969 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851601+0000", 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024058400000000001 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.856568+0000", 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049666449999999996 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103537 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.382481061, 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042298980999999999, 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103537 2024-09-13T05:00:08.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810406+0000", 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810409+0000", 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0489999999999999e-06 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810417+0000", 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.763e-06 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810423+0000", 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3779999999999998e-06 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852568+0000", 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042144632000000001 2024-09-13T05:00:08.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852644+0000", 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5908000000000006e-05 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852706+0000", 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1713999999999999e-05 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103538 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 484352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.382440778000003, 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042420402000000003, 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103538 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810446+0000", 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:08.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810457+0000", 2024-09-13T05:00:08.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0686999999999999e-05 2024-09-13T05:00:08.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810486+0000", 2024-09-13T05:00:08.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8784000000000001e-05 2024-09-13T05:00:08.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810494+0000", 2024-09-13T05:00:08.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5979999999999997e-06 2024-09-13T05:00:08.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852727+0000", 2024-09-13T05:00:08.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042232984000000001 2024-09-13T05:00:08.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852801+0000", 2024-09-13T05:00:08.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3691999999999998e-05 2024-09-13T05:00:08.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852867+0000", 2024-09-13T05:00:08.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6391999999999997e-05 2024-09-13T05:00:08.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.292 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103539 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.292 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:08.292 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.382385593000002, 2024-09-13T05:00:08.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042532474000000001, 2024-09-13T05:00:08.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.293 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103539 2024-09-13T05:00:08.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.294 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:08.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:08.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810501+0000", 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810506+0000", 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1740000000000002e-06 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810515+0000", 2024-09-13T05:00:08.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9099999999999994e-06 2024-09-13T05:00:08.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810521+0000", 2024-09-13T05:00:08.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5359999999999998e-06 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852890+0000", 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042368994 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852968+0000", 2024-09-13T05:00:08.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8097999999999997e-05 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853035+0000", 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6501999999999994e-05 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103540 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 510976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:08.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.382360687999999, 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042686832000000001, 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103540 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:08.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810530+0000", 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.88e-06 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810538+0000", 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1729999999999995e-06 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810544+0000", 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2179999999999996e-06 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853061+0000", 2024-09-13T05:00:08.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042516929000000002 2024-09-13T05:00:08.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853135+0000", 2024-09-13T05:00:08.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4333999999999997e-05 2024-09-13T05:00:08.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853214+0000", 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8715999999999996e-05 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.306 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:08.307 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:00:08.308 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:08.257+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:08.364 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:08.358+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:08.431 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:08.431 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:08.431 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T05:00:08.431 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:08.446 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:00:08.490 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:08.486+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:08.566 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:08.562+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:08.609 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:08.609 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:08.609 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T05:00:08.609 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T05:00:08.609 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:08.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:00:08.666 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:08.661+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:08.767 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:08.765+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:08.795 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 134.628885881, 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T05:00:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T05:00:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 112.626445222, 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T05:00:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 98.779613280999996, 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T05:00:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T05:00:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T05:00:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 98.765102229000007, 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T05:00:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T05:00:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99140 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.423378526, 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058811783999999999, 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99140 2024-09-13T05:00:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369483+0000", 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371901+0000", 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024177479999999999 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371935+0000", 2024-09-13T05:00:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999999e-05 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371949+0000", 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3917000000000001e-05 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372078+0000", 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012926999999999999 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372339+0000", 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000260394 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.373383+0000", 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010445809999999999 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428034+0000", 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054651138000000002 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428093+0000", 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8604000000000003e-05 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428253+0000", 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016013 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428297+0000", 2024-09-13T05:00:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4459000000000002e-05 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99141 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.420895094000002, 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058946757000000002, 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99141 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371967+0000", 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371974+0000", 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2609999999999999e-06 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371981+0000", 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371986+0000", 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3630000000000002e-06 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424374+0000", 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.052387356000000003 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424479+0000", 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010508199999999999 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424690+0000", 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021145300000000001 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429820+0000", 2024-09-13T05:00:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051295810000000002 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430829+0000", 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010095480000000001 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430878+0000", 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8979e-05 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430916+0000", 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7413000000000002e-05 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106233 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1172480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.671086531, 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054064492999999998, 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106233 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121777+0000", 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121793+0000", 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5945999999999999e-05 2024-09-13T05:00:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121798+0000", 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3030000000000001e-06 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121801+0000", 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.898e-06 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175652+0000", 2024-09-13T05:00:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053851796 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175743+0000", 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1002999999999996e-05 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175842+0000", 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8745000000000004e-05 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106244 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1191936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.671060660999999, 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054259089000000003, 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106244 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121803+0000", 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121805+0000", 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.53e-06 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121808+0000", 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.96e-06 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121810+0000", 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175871+0000", 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054060460999999997 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175956+0000", 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5227000000000002e-05 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176063+0000", 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106764 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106245 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1205248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.671051880999997, 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054466459000000002, 2024-09-13T05:00:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106245 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121814+0000", 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4079999999999999e-06 2024-09-13T05:00:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121816+0000", 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8880000000000001e-06 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121819+0000", 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176099+0000", 2024-09-13T05:00:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054279922000000001 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176176+0000", 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7495000000000003e-05 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176279+0000", 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102503 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106246 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1218560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.671043193999999, 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054660164999999997, 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106246 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121822+0000", 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.342e-06 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121825+0000", 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.005e-06 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121828+0000", 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.807e-06 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176305+0000", 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054476681999999998 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176387+0000", 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2238999999999996e-05 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176481+0000", 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4287e-05 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106247 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1231872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.671034083999999, 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054861114000000002, 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106247 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121831+0000", 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3209999999999999e-06 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121834+0000", 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121837+0000", 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:00:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176507+0000", 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054670165 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176591+0000", 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3820999999999996e-05 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176691+0000", 2024-09-13T05:00:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100739 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106258 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1384448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.671025381, 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055059518000000002, 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106258 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121840+0000", 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2890000000000001e-06 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121843+0000", 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8700000000000001e-06 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121845+0000", 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6869999999999999e-06 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176717+0000", 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054871862 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176801+0000", 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3473999999999994e-05 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176898+0000", 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7535999999999996e-05 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106259 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1397760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.671015316999998, 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055256324000000002, 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106259 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121851+0000", 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9920000000000002e-06 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121856+0000", 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5090000000000001e-06 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121861+0000", 2024-09-13T05:00:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176928+0000", 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055067656999999999 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177010+0000", 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1298000000000007e-05 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177105+0000", 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5623000000000002e-05 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106260 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.671000849999999, 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055448655999999999, 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106260 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121865+0000", 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.553e-06 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121868+0000", 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.106e-06 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121870+0000", 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.481e-06 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177133+0000", 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055262349000000002 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177212+0000", 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9305e-05 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177312+0000", 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100083 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106261 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1424384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.670991806000004, 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055627715000000001, 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106261 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121874+0000", 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3379999999999999e-06 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121876+0000", 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.937e-06 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121879+0000", 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4509999999999999e-06 2024-09-13T05:00:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177331+0000", 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055451584999999998 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177401+0000", 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0269000000000001e-05 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177500+0000", 2024-09-13T05:00:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9335999999999999e-05 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106262 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1437696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.670983057000001, 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055839664999999997, 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106262 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121883+0000", 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.823e-06 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121888+0000", 2024-09-13T05:00:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121893+0000", 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.425e-06 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177529+0000", 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05563651 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177616+0000", 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6794999999999998e-05 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177721+0000", 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104991 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106284 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1245184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.669556600999996, 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054629361000000001, 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106284 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123307+0000", 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123312+0000", 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9640000000000002e-06 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123324+0000", 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1965e-05 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123332+0000", 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0979999999999994e-06 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177750+0000", 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054418375999999997 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177832+0000", 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1947999999999995e-05 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177937+0000", 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010498299999999999 2024-09-13T05:00:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106285 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1258496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.669524758000001, 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054821169000000003, 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106285 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123342+0000", 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3519999999999998e-06 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123350+0000", 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6159999999999996e-06 2024-09-13T05:00:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123355+0000", 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2050000000000001e-06 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177969+0000", 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054613802000000003 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178053+0000", 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3800000000000004e-05 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178161+0000", 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107876 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106286 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.662629307000003, 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054693622999999997, 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106286 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130234+0000", 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130240+0000", 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1519999999999997e-06 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130255+0000", 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4739000000000001e-05 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130262+0000", 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5979999999999999e-06 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178190+0000", 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047928040999999998 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178285+0000", 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5082999999999997e-05 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.184929+0000", 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066430589999999998 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107657 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1654784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.581209768999997, 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054578494999999998, 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107657 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211657+0000", 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1920000000000001e-06 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211674+0000", 2024-09-13T05:00:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6923e-05 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211691+0000", 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6957e-05 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262566+0000", 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050875110000000001 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262650+0000", 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.266233+0000", 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035824989999999998 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:08.894 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:08.896 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:00:08.896 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:08.890+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:08.934 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T05:00:08.963 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T05:00:08.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:09.005 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:09.002+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:09.040 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:122357 3.19 3:9e9a16eb:::rbd_header.112aa73d3d8d:head [watch watch cookie 139644891971920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-13T05:00:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T05:00:09.037370+0000", 2024-09-13T05:00:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.0023069779999999999, 2024-09-13T05:00:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0023479070000000002, 2024-09-13T05:00:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.041 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 122357 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.037370+0000", 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.037370+0000", 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.037369+0000", 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.037375+0000", 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4909999999999996e-06 2024-09-13T05:00:09.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.037390+0000", 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5327000000000001e-05 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.037398+0000", 2024-09-13T05:00:09.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2409999999999995e-06 2024-09-13T05:00:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.037488+0000", 2024-09-13T05:00:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9219000000000001e-05 2024-09-13T05:00:09.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.037707+0000", 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021915699999999999 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 2", 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.038114+0000", 2024-09-13T05:00:09.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00040716800000000002 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039272+0000", 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0011577340000000001 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039400+0000", 2024-09-13T05:00:09.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000128035 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039527+0000", 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012704799999999999 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039718+0000", 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019145100000000001 2024-09-13T05:00:09.048 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_repop_reply(client.4394.0:122357 3.19 e126/105 ondisk, result = 0)", 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T05:00:09.039250+0000", 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.00042707200000000002, 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00064147599999999998, 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.049 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039250+0000", 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039250+0000", 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039249+0000", 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039255+0000", 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4870000000000002e-06 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039268+0000", 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3614e-05 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039276+0000", 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.058e-06 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039361+0000", 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5047000000000001e-05 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039383+0000", 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2846000000000001e-05 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:122358 3.19 3:9e9a16eb:::rbd_header.112aa73d3d8d:head [call lock.get_info in=30b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126)", 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T05:00:09.039763+0000", 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 4294967295.9999137, 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000241561, 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "reached pg", 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 122358 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039763+0000", 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039763+0000", 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039763+0000", 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039768+0000", 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1420000000000001e-06 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039780+0000", 2024-09-13T05:00:09.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1747e-05 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039786+0000", 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0700000000000003e-06 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:09.039940+0000", 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154785 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.058 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-13T05:00:09.059 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 3 2024-09-13T05:00:09.059 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:09.060 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:00:09.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:00:09.105 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:09.101+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:09.178 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:09.178 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:09.179 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:09.179 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:09.179 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:09.192 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:00:09.237 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:09.233+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:09.299 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:09.294+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:09.322 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:09.324 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:09.340 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-13T05:00:08.942168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":378462801,"num_objects":1008,"num_object_clones":181,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1008,"num_whiteouts":656,"num_read":160088,"num_read_kb":652062,"num_write":21027,"num_write_kb":344153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":71057662,"num_keys_recovered":178,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17663,"ondisk_log_size":17663,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":357780,"kb_used_data":29236,"kb_used_omap":10,"kb_used_meta":328437,"kb_avail":660245100,"statfs":{"total":676457349120,"available":676090982400,"internally_reserved":0,"allocated":29937664,"data_stored":196448564,"data_compressed":3712360,"data_compressed_allocated":14606336,"data_compressed_original":187717691,"omap_allocated":11149,"internal_metadata":336319603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-149127859,"num_objects":-400,"num_object_clones":-158,"num_object_copies":-800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-400,"num_whiteouts":-153,"num_read":1410,"num_read_kb":1206,"num_write":273,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.920368"},"pg_stats":[{"pgid":"3.1f","version":"123'552","reported_seq":3411,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.207217+0000","last_change":"2024-09-13T05:00:06.029272+0000","last_active":"2024-09-13T05:00:06.207217+0000","last_peered":"2024-09-13T05:00:06.207217+0000","last_clean":"2024-09-13T05:00:06.207217+0000","last_became_active":"2024-09-13T04:59:49.939066+0000","last_became_peered":"2024-09-13T04:59:49.939066+0000","last_unstale":"2024-09-13T05:00:06.207217+0000","last_undegraded":"2024-09-13T05:00:06.207217+0000","last_fullsized":"2024-09-13T05:00:06.207217+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:39.536667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014398939999999999,"stat_sum":{"num_bytes":26868257,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":23,"num_read":4215,"num_read_kb":16134,"num_write":573,"num_write_kb":9809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":7347311,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.1e","version":"123'377","reported_seq":2166,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.159581+0000","last_change":"2024-09-13T05:00:06.029915+0000","last_active":"2024-09-13T05:00:06.159581+0000","last_peered":"2024-09-13T05:00:06.159581+0000","last_clean":"2024-09-13T05:00:06.159581+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:06.159581+0000","last_undegraded":"2024-09-13T05:00:06.159581+0000","last_fullsized":"2024-09-13T05:00:06.159581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013133890000000001,"stat_sum":{"num_bytes":3604325,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":2755,"num_read_kb":18337,"num_write":333,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.1d","version":"124'519","reported_seq":3708,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087850+0000","last_change":"2024-09-13T05:00:07.087850+0000","last_active":"2024-09-13T05:00:07.087850+0000","last_peered":"2024-09-13T05:00:07.087850+0000","last_clean":"2024-09-13T05:00:07.087850+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:07.087850+0000","last_undegraded":"2024-09-13T05:00:07.087850+0000","last_fullsized":"2024-09-13T05:00:07.087850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160021,"stat_sum":{"num_bytes":20336804,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":4168,"num_read_kb":23676,"num_write":657,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1c","version":"124'578","reported_seq":1861,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.114431+0000","last_change":"2024-09-13T05:00:07.068194+0000","last_active":"2024-09-13T05:00:07.114431+0000","last_peered":"2024-09-13T05:00:07.114431+0000","last_clean":"2024-09-13T05:00:07.114431+0000","last_became_active":"2024-09-13T04:59:48.932372+0000","last_became_peered":"2024-09-13T04:59:48.932372+0000","last_unstale":"2024-09-13T05:00:07.114431+0000","last_undegraded":"2024-09-13T05:00:07.114431+0000","last_fullsized":"2024-09-13T05:00:07.114431+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:24.843964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064349000000000003,"stat_sum":{"num_bytes":7454720,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":2097,"num_read_kb":8182,"num_write":715,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":5244301,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1b","version":"122'500","reported_seq":5015,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048707+0000","last_change":"2024-09-13T05:00:07.049219+0000","last_active":"2024-09-13T05:00:07.048707+0000","last_peered":"2024-09-13T05:00:07.048707+0000","last_clean":"2024-09-13T05:00:07.048707+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:07.048707+0000","last_undegraded":"2024-09-13T05:00:07.048707+0000","last_fullsized":"2024-09-13T05:00:07.048707+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001125708,"stat_sum":{"num_bytes":12918784,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":5625,"num_read_kb":17585,"num_write":555,"num_write_kb":6766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1a","version":"122'480","reported_seq":3818,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.191350+0000","last_change":"2024-09-13T05:00:06.028415+0000","last_active":"2024-09-13T05:00:06.191350+0000","last_peered":"2024-09-13T05:00:06.191350+0000","last_clean":"2024-09-13T05:00:06.191350+0000","last_became_active":"2024-09-13T04:59:48.942009+0000","last_became_peered":"2024-09-13T04:59:48.942009+0000","last_unstale":"2024-09-13T05:00:06.191350+0000","last_undegraded":"2024-09-13T05:00:06.191350+0000","last_fullsized":"2024-09-13T05:00:06.191350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:38:34.898545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089277099999999999,"stat_sum":{"num_bytes":17659856,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":21,"num_read":4697,"num_read_kb":13741,"num_write":538,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":6543600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.19","version":"124'434","reported_seq":18907,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.049966+0000","last_change":"2024-09-13T05:00:07.049966+0000","last_active":"2024-09-13T05:00:07.049966+0000","last_peered":"2024-09-13T05:00:07.049966+0000","last_clean":"2024-09-13T05:00:07.049966+0000","last_became_active":"2024-09-13T04:59:48.941448+0000","last_became_peered":"2024-09-13T04:59:48.941448+0000","last_unstale":"2024-09-13T05:00:07.049966+0000","last_undegraded":"2024-09-13T05:00:07.049966+0000","last_fullsized":"2024-09-13T05:00:07.049966+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:02:00.880300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016055209999999999,"stat_sum":{"num_bytes":14492081,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":19059,"num_read_kb":95954,"num_write":479,"num_write_kb":19269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":16500093,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.18","version":"122'467","reported_seq":2965,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028609+0000","last_change":"2024-09-13T05:00:06.029147+0000","last_active":"2024-09-13T05:00:06.028609+0000","last_peered":"2024-09-13T05:00:06.028609+0000","last_clean":"2024-09-13T05:00:06.028609+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:06.028609+0000","last_undegraded":"2024-09-13T05:00:06.028609+0000","last_fullsized":"2024-09-13T05:00:06.028609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014909400000000001,"stat_sum":{"num_bytes":17188355,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":20,"num_read":3251,"num_read_kb":9829,"num_write":549,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.17","version":"121'312","reported_seq":3663,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.029225+0000","last_change":"2024-09-13T05:00:06.029482+0000","last_active":"2024-09-13T05:00:06.029225+0000","last_peered":"2024-09-13T05:00:06.029225+0000","last_clean":"2024-09-13T05:00:06.029225+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:06.029225+0000","last_undegraded":"2024-09-13T05:00:06.029225+0000","last_fullsized":"2024-09-13T05:00:06.029225+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001461216,"stat_sum":{"num_bytes":14360724,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":3215,"num_read_kb":21670,"num_write":303,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.16","version":"122'529","reported_seq":2504,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.041654+0000","last_change":"2024-09-13T05:00:07.042229+0000","last_active":"2024-09-13T05:00:07.041654+0000","last_peered":"2024-09-13T05:00:07.041654+0000","last_clean":"2024-09-13T05:00:07.041654+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:07.041654+0000","last_undegraded":"2024-09-13T05:00:07.041654+0000","last_fullsized":"2024-09-13T05:00:07.041654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064906100000000004,"stat_sum":{"num_bytes":6348907,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":3479,"num_read_kb":19817,"num_write":649,"num_write_kb":7274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.15","version":"122'406","reported_seq":1162,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.068221+0000","last_change":"2024-09-13T05:00:07.068378+0000","last_active":"2024-09-13T05:00:07.068221+0000","last_peered":"2024-09-13T05:00:07.068221+0000","last_clean":"2024-09-13T05:00:07.068221+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:07.068221+0000","last_undegraded":"2024-09-13T05:00:07.068221+0000","last_fullsized":"2024-09-13T05:00:07.068221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063102299999999998,"stat_sum":{"num_bytes":10107399,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1530,"num_read_kb":10660,"num_write":446,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.14","version":"124'698","reported_seq":4204,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087387+0000","last_change":"2024-09-13T05:00:07.087387+0000","last_active":"2024-09-13T05:00:07.087387+0000","last_peered":"2024-09-13T05:00:07.087387+0000","last_clean":"2024-09-13T05:00:07.087387+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:07.087387+0000","last_undegraded":"2024-09-13T05:00:07.087387+0000","last_fullsized":"2024-09-13T05:00:07.087387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083436199999999995,"stat_sum":{"num_bytes":7195972,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":5421,"num_read_kb":13519,"num_write":911,"num_write_kb":5068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":118,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.086491+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:07.086491+0000","last_peered":"2024-09-13T05:00:07.086491+0000","last_clean":"2024-09-13T05:00:07.086491+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:07.086491+0000","last_undegraded":"2024-09-13T05:00:07.086491+0000","last_fullsized":"2024-09-13T05:00:07.086491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"124'308","reported_seq":2566,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.187614+0000","last_change":"2024-09-13T05:00:07.103142+0000","last_active":"2024-09-13T05:00:07.187614+0000","last_peered":"2024-09-13T05:00:07.187614+0000","last_clean":"2024-09-13T05:00:07.187614+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:07.187614+0000","last_undegraded":"2024-09-13T05:00:07.187614+0000","last_fullsized":"2024-09-13T05:00:07.187614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116515,"stat_sum":{"num_bytes":18004187,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":23,"num_read":2154,"num_read_kb":23093,"num_write":280,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048249+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:07.048249+0000","last_peered":"2024-09-13T05:00:07.048249+0000","last_clean":"2024-09-13T05:00:07.048249+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:07.048249+0000","last_undegraded":"2024-09-13T05:00:07.048249+0000","last_fullsized":"2024-09-13T05:00:07.048249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"124'715","reported_seq":2032,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.049955+0000","last_change":"2024-09-13T05:00:07.049955+0000","last_active":"2024-09-13T05:00:07.049955+0000","last_peered":"2024-09-13T05:00:07.049955+0000","last_clean":"2024-09-13T05:00:07.049955+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:07.049955+0000","last_undegraded":"2024-09-13T05:00:07.049955+0000","last_fullsized":"2024-09-13T05:00:07.049955+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016657849999999999,"stat_sum":{"num_bytes":6287243,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":17,"num_read":4240,"num_read_kb":12408,"num_write":980,"num_write_kb":8392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093521+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:06.093521+0000","last_peered":"2024-09-13T05:00:06.093521+0000","last_clean":"2024-09-13T05:00:06.093521+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:06.093521+0000","last_undegraded":"2024-09-13T05:00:06.093521+0000","last_fullsized":"2024-09-13T05:00:06.093521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"122'621","reported_seq":1698,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028145+0000","last_change":"2024-09-13T05:00:06.028461+0000","last_active":"2024-09-13T05:00:06.028145+0000","last_peered":"2024-09-13T05:00:06.028145+0000","last_clean":"2024-09-13T05:00:06.028145+0000","last_became_active":"2024-09-13T04:59:29.177938+0000","last_became_peered":"2024-09-13T04:59:29.177938+0000","last_unstale":"2024-09-13T05:00:06.028145+0000","last_undegraded":"2024-09-13T05:00:06.028145+0000","last_fullsized":"2024-09-13T05:00:06.028145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:09.883322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085147300000000005,"stat_sum":{"num_bytes":6578176,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":29,"num_read":3117,"num_read_kb":14959,"num_write":796,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":117,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T05:00:05.024790+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:05.024790+0000","last_peered":"2024-09-13T05:00:05.024790+0000","last_clean":"2024-09-13T05:00:05.024790+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:05.024790+0000","last_undegraded":"2024-09-13T05:00:05.024790+0000","last_fullsized":"2024-09-13T05:00:05.024790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"122'536","reported_seq":2332,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.041416+0000","last_change":"2024-09-13T05:00:07.041865+0000","last_active":"2024-09-13T05:00:07.041416+0000","last_peered":"2024-09-13T05:00:07.041416+0000","last_clean":"2024-09-13T05:00:07.041416+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:07.041416+0000","last_undegraded":"2024-09-13T05:00:07.041416+0000","last_fullsized":"2024-09-13T05:00:07.041416+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055185599999999998,"stat_sum":{"num_bytes":2982045,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":3813,"num_read_kb":10813,"num_write":572,"num_write_kb":6332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.3","version":"124'833","reported_seq":4650,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.227435+0000","last_change":"2024-09-13T05:00:07.067852+0000","last_active":"2024-09-13T05:00:07.227435+0000","last_peered":"2024-09-13T05:00:07.227435+0000","last_clean":"2024-09-13T05:00:07.227435+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:07.227435+0000","last_undegraded":"2024-09-13T05:00:07.227435+0000","last_fullsized":"2024-09-13T05:00:07.227435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042328299999999999,"stat_sum":{"num_bytes":15830567,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":23,"num_read":6963,"num_read_kb":22566,"num_write":910,"num_write_kb":15653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.2","version":"23'2","reported_seq":120,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093935+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:06.093935+0000","last_peered":"2024-09-13T05:00:06.093935+0000","last_clean":"2024-09-13T05:00:06.093935+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:06.093935+0000","last_undegraded":"2024-09-13T05:00:06.093935+0000","last_fullsized":"2024-09-13T05:00:06.093935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"122'1196","reported_seq":2773,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028836+0000","last_change":"2024-09-13T05:00:06.028836+0000","last_active":"2024-09-13T05:00:06.028836+0000","last_peered":"2024-09-13T05:00:06.028836+0000","last_clean":"2024-09-13T05:00:06.028836+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:06.028836+0000","last_undegraded":"2024-09-13T05:00:06.028836+0000","last_fullsized":"2024-09-13T05:00:06.028836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1196,"log_dups_size":0,"ondisk_log_size":1196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220477,"stat_sum":{"num_bytes":4067826,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":23,"num_read":7005,"num_read_kb":20796,"num_write":1817,"num_write_kb":11791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.056888+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T05:00:06.056888+0000","last_peered":"2024-09-13T05:00:06.056888+0000","last_clean":"2024-09-13T05:00:06.056888+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T05:00:06.056888+0000","last_undegraded":"2024-09-13T05:00:06.056888+0000","last_fullsized":"2024-09-13T05:00:06.056888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"124'441","reported_seq":5059,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.102091+0000","last_change":"2024-09-13T05:00:07.102091+0000","last_active":"2024-09-13T05:00:07.102091+0000","last_peered":"2024-09-13T05:00:07.102091+0000","last_clean":"2024-09-13T05:00:07.102091+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:07.102091+0000","last_undegraded":"2024-09-13T05:00:07.102091+0000","last_fullsized":"2024-09-13T05:00:07.102091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065723400000000003,"stat_sum":{"num_bytes":28878669,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":5759,"num_read_kb":23227,"num_write":423,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093118+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:06.093118+0000","last_peered":"2024-09-13T05:00:06.093118+0000","last_clean":"2024-09-13T05:00:06.093118+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:06.093118+0000","last_undegraded":"2024-09-13T05:00:06.093118+0000","last_fullsized":"2024-09-13T05:00:06.093118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'1026","reported_seq":3238,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.215709+0000","last_change":"2024-09-13T05:00:07.052184+0000","last_active":"2024-09-13T05:00:07.215709+0000","last_peered":"2024-09-13T05:00:07.215709+0000","last_clean":"2024-09-13T05:00:07.215709+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:07.215709+0000","last_undegraded":"2024-09-13T05:00:07.215709+0000","last_fullsized":"2024-09-13T05:00:07.215709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003741753,"stat_sum":{"num_bytes":2686992,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":25,"num_read":6899,"num_read_kb":25000,"num_write":1435,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.3","version":"21'1","reported_seq":119,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.094307+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:06.094307+0000","last_peered":"2024-09-13T05:00:06.094307+0000","last_clean":"2024-09-13T05:00:06.094307+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:06.094307+0000","last_undegraded":"2024-09-13T05:00:06.094307+0000","last_fullsized":"2024-09-13T05:00:06.094307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":201,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T05:00:05.024942+0000","last_change":"2024-09-13T04:59:48.940179+0000","last_active":"2024-09-13T05:00:05.024942+0000","last_peered":"2024-09-13T05:00:05.024942+0000","last_clean":"2024-09-13T05:00:05.024942+0000","last_became_active":"2024-09-13T04:59:48.938034+0000","last_became_peered":"2024-09-13T04:59:48.938034+0000","last_unstale":"2024-09-13T05:00:05.024942+0000","last_undegraded":"2024-09-13T05:00:05.024942+0000","last_fullsized":"2024-09-13T05:00:05.024942+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:51:02.403769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"122'485","reported_seq":3725,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048553+0000","last_change":"2024-09-13T05:00:07.049123+0000","last_active":"2024-09-13T05:00:07.048553+0000","last_peered":"2024-09-13T05:00:07.048553+0000","last_clean":"2024-09-13T05:00:07.048553+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:07.048553+0000","last_undegraded":"2024-09-13T05:00:07.048553+0000","last_fullsized":"2024-09-13T05:00:07.048553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244079999999999,"stat_sum":{"num_bytes":2158608,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":4652,"num_read_kb":13059,"num_write":514,"num_write_kb":10371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.9","version":"124'579","reported_seq":5925,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.257562+0000","last_change":"2024-09-13T05:00:07.087289+0000","last_active":"2024-09-13T05:00:07.257562+0000","last_peered":"2024-09-13T05:00:07.257562+0000","last_clean":"2024-09-13T05:00:07.257562+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:07.257562+0000","last_undegraded":"2024-09-13T05:00:07.257562+0000","last_fullsized":"2024-09-13T05:00:07.257562+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072982600000000004,"stat_sum":{"num_bytes":17874913,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":6863,"num_read_kb":22408,"num_write":702,"num_write_kb":12026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.a","version":"124'462","reported_seq":6936,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.190543+0000","last_change":"2024-09-13T05:00:07.087704+0000","last_active":"2024-09-13T05:00:07.190543+0000","last_peered":"2024-09-13T05:00:07.190543+0000","last_clean":"2024-09-13T05:00:07.190543+0000","last_became_active":"2024-09-13T04:59:48.935630+0000","last_became_peered":"2024-09-13T04:59:48.935630+0000","last_unstale":"2024-09-13T05:00:07.190543+0000","last_undegraded":"2024-09-13T05:00:07.190543+0000","last_fullsized":"2024-09-13T05:00:07.190543+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:27:40.051698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172389999999999,"stat_sum":{"num_bytes":12781457,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":6351,"num_read_kb":24124,"num_write":452,"num_write_kb":14043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":8097453,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.b","version":"124'319","reported_seq":2076,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.197090+0000","last_change":"2024-09-13T05:00:07.057557+0000","last_active":"2024-09-13T05:00:07.197090+0000","last_peered":"2024-09-13T05:00:07.197090+0000","last_clean":"2024-09-13T05:00:07.197090+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T05:00:07.197090+0000","last_undegraded":"2024-09-13T05:00:07.197090+0000","last_fullsized":"2024-09-13T05:00:07.197090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073781599999999997,"stat_sum":{"num_bytes":10190542,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":1663,"num_read_kb":17144,"num_write":300,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.c","version":"124'658","reported_seq":1805,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.186539+0000","last_change":"2024-09-13T05:00:07.068140+0000","last_active":"2024-09-13T05:00:07.186539+0000","last_peered":"2024-09-13T05:00:07.186539+0000","last_clean":"2024-09-13T05:00:07.186539+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:07.186539+0000","last_undegraded":"2024-09-13T05:00:07.186539+0000","last_fullsized":"2024-09-13T05:00:07.186539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050180799999999996,"stat_sum":{"num_bytes":8858126,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":24,"num_read":3102,"num_read_kb":9307,"num_write":850,"num_write_kb":7831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.d","version":"123'435","reported_seq":4594,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087931+0000","last_change":"2024-09-13T05:00:07.088168+0000","last_active":"2024-09-13T05:00:07.087931+0000","last_peered":"2024-09-13T05:00:07.087931+0000","last_clean":"2024-09-13T05:00:07.087931+0000","last_became_active":"2024-09-13T04:59:48.931622+0000","last_became_peered":"2024-09-13T04:59:48.931622+0000","last_unstale":"2024-09-13T05:00:07.087931+0000","last_undegraded":"2024-09-13T05:00:07.087931+0000","last_fullsized":"2024-09-13T05:00:07.087931+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:17:05.641262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001391818,"stat_sum":{"num_bytes":7429674,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":5605,"num_read_kb":35524,"num_write":608,"num_write_kb":15014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":11706466,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.e","version":"122'420","reported_seq":1883,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.042027+0000","last_change":"2024-09-13T05:00:07.042282+0000","last_active":"2024-09-13T05:00:07.042027+0000","last_peered":"2024-09-13T05:00:07.042027+0000","last_clean":"2024-09-13T05:00:07.042027+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:07.042027+0000","last_undegraded":"2024-09-13T05:00:07.042027+0000","last_fullsized":"2024-09-13T05:00:07.042027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057327599999999995,"stat_sum":{"num_bytes":10108580,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":2827,"num_read_kb":9044,"num_write":390,"num_write_kb":5098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.f","version":"124'824","reported_seq":8416,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.254680+0000","last_change":"2024-09-13T05:00:07.068576+0000","last_active":"2024-09-13T05:00:07.254680+0000","last_peered":"2024-09-13T05:00:07.254680+0000","last_clean":"2024-09-13T05:00:07.254680+0000","last_became_active":"2024-09-13T04:59:48.941958+0000","last_became_peered":"2024-09-13T04:59:48.941958+0000","last_unstale":"2024-09-13T05:00:07.254680+0000","last_undegraded":"2024-09-13T05:00:07.254680+0000","last_fullsized":"2024-09-13T05:00:07.254680+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:09:58.793384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071082000000000003,"stat_sum":{"num_bytes":14807155,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":23,"num_read":9762,"num_read_kb":27337,"num_write":1138,"num_write_kb":19486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":7326157,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.10","version":"122'423","reported_seq":7043,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.027979+0000","last_change":"2024-09-13T05:00:06.027979+0000","last_active":"2024-09-13T05:00:06.027979+0000","last_peered":"2024-09-13T05:00:06.027979+0000","last_clean":"2024-09-13T05:00:06.027979+0000","last_became_active":"2024-09-13T04:59:48.932014+0000","last_became_peered":"2024-09-13T04:59:48.932014+0000","last_unstale":"2024-09-13T05:00:06.027979+0000","last_undegraded":"2024-09-13T05:00:06.027979+0000","last_fullsized":"2024-09-13T05:00:06.027979+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:12:17.905901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055876299999999999,"stat_sum":{"num_bytes":13830928,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":32,"num_read":7202,"num_read_kb":16550,"num_write":418,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2709903,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.11","version":"124'392","reported_seq":5074,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.187345+0000","last_change":"2024-09-13T05:00:07.087052+0000","last_active":"2024-09-13T05:00:07.187345+0000","last_peered":"2024-09-13T05:00:07.187345+0000","last_clean":"2024-09-13T05:00:07.187345+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:07.187345+0000","last_undegraded":"2024-09-13T05:00:07.187345+0000","last_fullsized":"2024-09-13T05:00:07.187345+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000684714,"stat_sum":{"num_bytes":22811071,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":5435,"num_read_kb":31446,"num_write":396,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.12","version":"124'432","reported_seq":3586,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.102474+0000","last_change":"2024-09-13T05:00:07.102474+0000","last_active":"2024-09-13T05:00:07.102474+0000","last_peered":"2024-09-13T05:00:07.102474+0000","last_clean":"2024-09-13T05:00:07.102474+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:07.102474+0000","last_undegraded":"2024-09-13T05:00:07.102474+0000","last_fullsized":"2024-09-13T05:00:07.102474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085298500000000001,"stat_sum":{"num_bytes":4570155,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":18,"num_read":3947,"num_read_kb":13255,"num_write":420,"num_write_kb":7303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.13","version":"122'586","reported_seq":1824,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.086833+0000","last_change":"2024-09-13T05:00:07.087037+0000","last_active":"2024-09-13T05:00:07.086833+0000","last_peered":"2024-09-13T05:00:07.086833+0000","last_clean":"2024-09-13T05:00:07.086833+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:07.086833+0000","last_undegraded":"2024-09-13T05:00:07.086833+0000","last_fullsized":"2024-09-13T05:00:07.086833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053792499999999997,"stat_sum":{"num_bytes":8599316,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":3101,"num_read_kb":10798,"num_write":757,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":377872414,"num_objects":1004,"num_object_clones":181,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":656,"num_read":159972,"num_read_kb":651962,"num_write":20866,"num_write_kb":341735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":954,"num_bytes_recovered":70467294,"num_keys_recovered":178,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27303936,"data_stored":248052570,"data_compressed":6026891,"data_compressed_allocated":20779008,"data_compressed_original":243111143,"omap_allocated":0,"internal_metadata":0},"log_size":17543,"ondisk_log_size":17543,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":19544,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43016,"kb_used_data":3704,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94328824,"statfs":{"total":96636764160,"available":96592715776,"internally_reserved":0,"allocated":3792896,"data_stored":22261128,"data_compressed":409733,"data_compressed_allocated":1712128,"data_compressed_original":21094400,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":1.1419999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.92700000000000005}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.054},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.97099999999999997}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":1.1020000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.1859999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30772,"kb_used_data":3748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341068,"statfs":{"total":96636764160,"available":96605253632,"internally_reserved":0,"allocated":3837952,"data_stored":22990142,"data_compressed":314142,"data_compressed_allocated":1712128,"data_compressed_original":21816215,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":0.873},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.96999999999999997}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.05}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.165},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.0129999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.141},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.0820000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53028,"kb_used_data":3864,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318812,"statfs":{"total":96636764160,"available":96582463488,"internally_reserved":0,"allocated":3956736,"data_stored":25404186,"data_compressed":428709,"data_compressed_allocated":1839104,"data_compressed_original":24204698,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.2490000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.0409999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1910000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0860000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58528,"kb_used_data":5268,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94313312,"statfs":{"total":96636764160,"available":96576831488,"internally_reserved":0,"allocated":5394432,"data_stored":38411852,"data_compressed":811081,"data_compressed_allocated":2904064,"data_compressed_original":36913040,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.1539999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84999999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.228},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.012},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.1910000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61472,"kb_used_data":4112,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94310368,"statfs":{"total":96636764160,"available":96573816832,"internally_reserved":0,"allocated":4210688,"data_stored":29755378,"data_compressed":645184,"data_compressed_allocated":2179072,"data_compressed_original":28658130,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.378},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.88}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.66800000000000004}]}]},{"osd":1,"up_from":0,"seq":64424509471,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57388,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94314452,"statfs":{"total":96636764160,"available":96577998848,"internally_reserved":0,"allocated":4227072,"data_stored":28900865,"data_compressed":543997,"data_compressed_allocated":2084864,"data_compressed_original":27691912,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.28399999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.501},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.753}]}]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53576,"kb_used_data":4412,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318264,"statfs":{"total":96636764160,"available":96581902336,"internally_reserved":0,"allocated":4517888,"data_stored":28725013,"data_compressed":559514,"data_compressed_allocated":2174976,"data_compressed_original":27339296,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.89400000000000002}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2752512,"data_stored":28207094,"data_compressed":543997,"data_compressed_allocated":2084864,"data_compressed_original":27691912,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8847360,"data_stored":68621506,"data_compressed":2518998,"data_compressed_allocated":7094272,"data_compressed_original":67349676,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2695168,"data_stored":28471239,"data_compressed":638094,"data_compressed_allocated":2142208,"data_compressed_original":28068306,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3297280,"data_stored":31296936,"data_compressed":634508,"data_compressed_allocated":2359296,"data_compressed_original":30560144,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2482176,"data_stored":24710415,"data_compressed":428709,"data_compressed_allocated":1839104,"data_compressed_original":24204698,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2998272,"data_stored":27447138,"data_compressed":553287,"data_compressed_allocated":2138112,"data_compressed_original":26765856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2359296,"data_stored":22296352,"data_compressed":314142,"data_compressed_allocated":1712128,"data_compressed_original":21816215,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1871872,"data_stored":17001890,"data_compressed":395156,"data_compressed_allocated":1409024,"data_compressed_original":16654336,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:09.345 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:09.360 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:09.360 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.499244316000002, 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T05:00:09.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T05:00:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T05:00:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T05:00:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T05:00:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.499217404999996, 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.365 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T05:00:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.366 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.367 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T05:00:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T05:00:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:09.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T05:00:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T05:00:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.369 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T05:00:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T05:00:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T05:00:09.370 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T05:00:09.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.371 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T05:00:09.371 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T05:00:09.371 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.371 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.371 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.372 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.495471769000005, 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T05:00:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.375 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T05:00:09.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T05:00:09.376 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.376 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.376 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.376 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T05:00:09.376 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T05:00:09.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.377 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T05:00:09.377 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T05:00:09.377 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.377 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.377 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.378 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.379 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T05:00:09.379 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T05:00:09.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.379 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.379 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.379 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.379 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.379 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.380 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:09.380 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.491928443999996, 2024-09-13T05:00:09.380 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T05:00:09.380 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.380 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.380 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.381 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:09.382 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.382 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.382 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.382 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.382 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.383 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.384 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.385 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.491888815999999, 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:09.386 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.387 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.388 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T05:00:09.389 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.491865156000003, 2024-09-13T05:00:09.390 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.391 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T05:00:09.392 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T05:00:09.393 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.394 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.491777607000003, 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.395 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.396 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.397 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.398 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.491724950999995, 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.400 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.401 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T05:00:09.402 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T05:00:09.403 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.48779906, 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.404 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.405 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.406 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.407 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.408 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.487773434999994, 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.409 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.410 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T05:00:09.411 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.412 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.487740040999995, 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.413 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.414 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.415 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T05:00:09.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:00:09.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.416 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.416 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.416 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T05:00:09.416 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.417 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.418 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.482950111999997, 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T05:00:09.419 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.420 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.421 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.422 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.423 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.479555482999999, 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.424 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.426 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T05:00:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.479482806999997, 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T05:00:09.428 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.429 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.430 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.431 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T05:00:09.432 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.475740446000003, 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.433 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:09.434 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T05:00:09.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T05:00:09.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.475567455000004, 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.438 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T05:00:09.440 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.441 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:09.442 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.471724855999994, 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:09.444 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.444 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.444 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:09.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.445 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.445 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T05:00:09.445 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T05:00:09.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.446 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.446 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T05:00:09.446 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T05:00:09.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T05:00:09.447 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T05:00:09.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.447 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.447 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T05:00:09.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T05:00:09.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.448 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.448 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T05:00:09.448 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T05:00:09.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.449 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.449 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T05:00:09.449 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T05:00:09.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.449 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.450 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.450 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:09.450 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.431876877999997, 2024-09-13T05:00:09.450 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T05:00:09.450 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.451 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.451 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.451 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T05:00:09.451 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.451 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.452 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.452 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:09.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.452 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:09.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:09.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T05:00:09.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T05:00:09.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T05:00:09.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T05:00:09.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T05:00:09.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T05:00:09.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.457 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T05:00:09.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T05:00:09.458 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.458 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.458 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.458 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.427547656000002, 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T05:00:09.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T05:00:09.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T05:00:09.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:09.463 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 75.419818656000004, 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.467 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T05:00:09.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T05:00:09.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.468 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.468 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.468 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:09.470 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:00:09.470 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:09.466+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:09.570 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:09.565+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:09.596 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:09.596 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:09.597 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:09.611 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:00:09.697 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:09.693+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:09.772 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:09.769+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:09.779 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:09.780 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:09.781 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:09.781 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:09.781 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:09.781 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:09.781 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:09.793 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:00:09.799 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-13T05:00:08.942168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":378462801,"num_objects":1008,"num_object_clones":181,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1008,"num_whiteouts":656,"num_read":160088,"num_read_kb":652062,"num_write":21027,"num_write_kb":344153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":71057662,"num_keys_recovered":178,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17663,"ondisk_log_size":17663,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":357780,"kb_used_data":29236,"kb_used_omap":10,"kb_used_meta":328437,"kb_avail":660245100,"statfs":{"total":676457349120,"available":676090982400,"internally_reserved":0,"allocated":29937664,"data_stored":196448564,"data_compressed":3712360,"data_compressed_allocated":14606336,"data_compressed_original":187717691,"omap_allocated":11149,"internal_metadata":336319603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-149127859,"num_objects":-400,"num_object_clones":-158,"num_object_copies":-800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-400,"num_whiteouts":-153,"num_read":1410,"num_read_kb":1206,"num_write":273,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.920368"},"pg_stats":[{"pgid":"3.1f","version":"123'552","reported_seq":3411,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.207217+0000","last_change":"2024-09-13T05:00:06.029272+0000","last_active":"2024-09-13T05:00:06.207217+0000","last_peered":"2024-09-13T05:00:06.207217+0000","last_clean":"2024-09-13T05:00:06.207217+0000","last_became_active":"2024-09-13T04:59:49.939066+0000","last_became_peered":"2024-09-13T04:59:49.939066+0000","last_unstale":"2024-09-13T05:00:06.207217+0000","last_undegraded":"2024-09-13T05:00:06.207217+0000","last_fullsized":"2024-09-13T05:00:06.207217+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:39.536667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014398939999999999,"stat_sum":{"num_bytes":26868257,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":23,"num_read":4215,"num_read_kb":16134,"num_write":573,"num_write_kb":9809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":7347311,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.1e","version":"123'377","reported_seq":2166,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.159581+0000","last_change":"2024-09-13T05:00:06.029915+0000","last_active":"2024-09-13T05:00:06.159581+0000","last_peered":"2024-09-13T05:00:06.159581+0000","last_clean":"2024-09-13T05:00:06.159581+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:06.159581+0000","last_undegraded":"2024-09-13T05:00:06.159581+0000","last_fullsized":"2024-09-13T05:00:06.159581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013133890000000001,"stat_sum":{"num_bytes":3604325,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":2755,"num_read_kb":18337,"num_write":333,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.1d","version":"124'519","reported_seq":3708,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087850+0000","last_change":"2024-09-13T05:00:07.087850+0000","last_active":"2024-09-13T05:00:07.087850+0000","last_peered":"2024-09-13T05:00:07.087850+0000","last_clean":"2024-09-13T05:00:07.087850+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:07.087850+0000","last_undegraded":"2024-09-13T05:00:07.087850+0000","last_fullsized":"2024-09-13T05:00:07.087850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160021,"stat_sum":{"num_bytes":20336804,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":4168,"num_read_kb":23676,"num_write":657,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1c","version":"124'578","reported_seq":1861,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.114431+0000","last_change":"2024-09-13T05:00:07.068194+0000","last_active":"2024-09-13T05:00:07.114431+0000","last_peered":"2024-09-13T05:00:07.114431+0000","last_clean":"2024-09-13T05:00:07.114431+0000","last_became_active":"2024-09-13T04:59:48.932372+0000","last_became_peered":"2024-09-13T04:59:48.932372+0000","last_unstale":"2024-09-13T05:00:07.114431+0000","last_undegraded":"2024-09-13T05:00:07.114431+0000","last_fullsized":"2024-09-13T05:00:07.114431+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:24.843964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064349000000000003,"stat_sum":{"num_bytes":7454720,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":2097,"num_read_kb":8182,"num_write":715,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":5244301,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1b","version":"122'500","reported_seq":5015,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048707+0000","last_change":"2024-09-13T05:00:07.049219+0000","last_active":"2024-09-13T05:00:07.048707+0000","last_peered":"2024-09-13T05:00:07.048707+0000","last_clean":"2024-09-13T05:00:07.048707+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:07.048707+0000","last_undegraded":"2024-09-13T05:00:07.048707+0000","last_fullsized":"2024-09-13T05:00:07.048707+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001125708,"stat_sum":{"num_bytes":12918784,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":5625,"num_read_kb":17585,"num_write":555,"num_write_kb":6766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1a","version":"122'480","reported_seq":3818,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.191350+0000","last_change":"2024-09-13T05:00:06.028415+0000","last_active":"2024-09-13T05:00:06.191350+0000","last_peered":"2024-09-13T05:00:06.191350+0000","last_clean":"2024-09-13T05:00:06.191350+0000","last_became_active":"2024-09-13T04:59:48.942009+0000","last_became_peered":"2024-09-13T04:59:48.942009+0000","last_unstale":"2024-09-13T05:00:06.191350+0000","last_undegraded":"2024-09-13T05:00:06.191350+0000","last_fullsized":"2024-09-13T05:00:06.191350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:38:34.898545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089277099999999999,"stat_sum":{"num_bytes":17659856,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":21,"num_read":4697,"num_read_kb":13741,"num_write":538,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":6543600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.19","version":"124'434","reported_seq":18907,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.049966+0000","last_change":"2024-09-13T05:00:07.049966+0000","last_active":"2024-09-13T05:00:07.049966+0000","last_peered":"2024-09-13T05:00:07.049966+0000","last_clean":"2024-09-13T05:00:07.049966+0000","last_became_active":"2024-09-13T04:59:48.941448+0000","last_became_peered":"2024-09-13T04:59:48.941448+0000","last_unstale":"2024-09-13T05:00:07.049966+0000","last_undegraded":"2024-09-13T05:00:07.049966+0000","last_fullsized":"2024-09-13T05:00:07.049966+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:02:00.880300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016055209999999999,"stat_sum":{"num_bytes":14492081,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":19059,"num_read_kb":95954,"num_write":479,"num_write_kb":19269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":16500093,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.18","version":"122'467","reported_seq":2965,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028609+0000","last_change":"2024-09-13T05:00:06.029147+0000","last_active":"2024-09-13T05:00:06.028609+0000","last_peered":"2024-09-13T05:00:06.028609+0000","last_clean":"2024-09-13T05:00:06.028609+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:06.028609+0000","last_undegraded":"2024-09-13T05:00:06.028609+0000","last_fullsized":"2024-09-13T05:00:06.028609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014909400000000001,"stat_sum":{"num_bytes":17188355,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":20,"num_read":3251,"num_read_kb":9829,"num_write":549,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.17","version":"121'312","reported_seq":3663,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.029225+0000","last_change":"2024-09-13T05:00:06.029482+0000","last_active":"2024-09-13T05:00:06.029225+0000","last_peered":"2024-09-13T05:00:06.029225+0000","last_clean":"2024-09-13T05:00:06.029225+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:06.029225+0000","last_undegraded":"2024-09-13T05:00:06.029225+0000","last_fullsized":"2024-09-13T05:00:06.029225+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001461216,"stat_sum":{"num_bytes":14360724,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":3215,"num_read_kb":21670,"num_write":303,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.16","version":"122'529","reported_seq":2504,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.041654+0000","last_change":"2024-09-13T05:00:07.042229+0000","last_active":"2024-09-13T05:00:07.041654+0000","last_peered":"2024-09-13T05:00:07.041654+0000","last_clean":"2024-09-13T05:00:07.041654+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:07.041654+0000","last_undegraded":"2024-09-13T05:00:07.041654+0000","last_fullsized":"2024-09-13T05:00:07.041654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064906100000000004,"stat_sum":{"num_bytes":6348907,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":3479,"num_read_kb":19817,"num_write":649,"num_write_kb":7274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.15","version":"122'406","reported_seq":1162,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.068221+0000","last_change":"2024-09-13T05:00:07.068378+0000","last_active":"2024-09-13T05:00:07.068221+0000","last_peered":"2024-09-13T05:00:07.068221+0000","last_clean":"2024-09-13T05:00:07.068221+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:07.068221+0000","last_undegraded":"2024-09-13T05:00:07.068221+0000","last_fullsized":"2024-09-13T05:00:07.068221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063102299999999998,"stat_sum":{"num_bytes":10107399,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1530,"num_read_kb":10660,"num_write":446,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.14","version":"124'698","reported_seq":4204,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087387+0000","last_change":"2024-09-13T05:00:07.087387+0000","last_active":"2024-09-13T05:00:07.087387+0000","last_peered":"2024-09-13T05:00:07.087387+0000","last_clean":"2024-09-13T05:00:07.087387+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:07.087387+0000","last_undegraded":"2024-09-13T05:00:07.087387+0000","last_fullsized":"2024-09-13T05:00:07.087387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083436199999999995,"stat_sum":{"num_bytes":7195972,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":5421,"num_read_kb":13519,"num_write":911,"num_write_kb":5068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":118,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.086491+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:07.086491+0000","last_peered":"2024-09-13T05:00:07.086491+0000","last_clean":"2024-09-13T05:00:07.086491+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:07.086491+0000","last_undegraded":"2024-09-13T05:00:07.086491+0000","last_fullsized":"2024-09-13T05:00:07.086491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"124'308","reported_seq":2566,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.187614+0000","last_change":"2024-09-13T05:00:07.103142+0000","last_active":"2024-09-13T05:00:07.187614+0000","last_peered":"2024-09-13T05:00:07.187614+0000","last_clean":"2024-09-13T05:00:07.187614+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:07.187614+0000","last_undegraded":"2024-09-13T05:00:07.187614+0000","last_fullsized":"2024-09-13T05:00:07.187614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116515,"stat_sum":{"num_bytes":18004187,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":23,"num_read":2154,"num_read_kb":23093,"num_write":280,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048249+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:07.048249+0000","last_peered":"2024-09-13T05:00:07.048249+0000","last_clean":"2024-09-13T05:00:07.048249+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:07.048249+0000","last_undegraded":"2024-09-13T05:00:07.048249+0000","last_fullsized":"2024-09-13T05:00:07.048249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"124'715","reported_seq":2032,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.049955+0000","last_change":"2024-09-13T05:00:07.049955+0000","last_active":"2024-09-13T05:00:07.049955+0000","last_peered":"2024-09-13T05:00:07.049955+0000","last_clean":"2024-09-13T05:00:07.049955+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:07.049955+0000","last_undegraded":"2024-09-13T05:00:07.049955+0000","last_fullsized":"2024-09-13T05:00:07.049955+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016657849999999999,"stat_sum":{"num_bytes":6287243,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":17,"num_read":4240,"num_read_kb":12408,"num_write":980,"num_write_kb":8392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093521+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:06.093521+0000","last_peered":"2024-09-13T05:00:06.093521+0000","last_clean":"2024-09-13T05:00:06.093521+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:06.093521+0000","last_undegraded":"2024-09-13T05:00:06.093521+0000","last_fullsized":"2024-09-13T05:00:06.093521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"122'621","reported_seq":1698,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028145+0000","last_change":"2024-09-13T05:00:06.028461+0000","last_active":"2024-09-13T05:00:06.028145+0000","last_peered":"2024-09-13T05:00:06.028145+0000","last_clean":"2024-09-13T05:00:06.028145+0000","last_became_active":"2024-09-13T04:59:29.177938+0000","last_became_peered":"2024-09-13T04:59:29.177938+0000","last_unstale":"2024-09-13T05:00:06.028145+0000","last_undegraded":"2024-09-13T05:00:06.028145+0000","last_fullsized":"2024-09-13T05:00:06.028145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:09.883322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085147300000000005,"stat_sum":{"num_bytes":6578176,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":29,"num_read":3117,"num_read_kb":14959,"num_write":796,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":117,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T05:00:05.024790+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:05.024790+0000","last_peered":"2024-09-13T05:00:05.024790+0000","last_clean":"2024-09-13T05:00:05.024790+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:05.024790+0000","last_undegraded":"2024-09-13T05:00:05.024790+0000","last_fullsized":"2024-09-13T05:00:05.024790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"122'536","reported_seq":2332,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.041416+0000","last_change":"2024-09-13T05:00:07.041865+0000","last_active":"2024-09-13T05:00:07.041416+0000","last_peered":"2024-09-13T05:00:07.041416+0000","last_clean":"2024-09-13T05:00:07.041416+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:07.041416+0000","last_undegraded":"2024-09-13T05:00:07.041416+0000","last_fullsized":"2024-09-13T05:00:07.041416+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055185599999999998,"stat_sum":{"num_bytes":2982045,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":3813,"num_read_kb":10813,"num_write":572,"num_write_kb":6332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.3","version":"124'833","reported_seq":4650,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.227435+0000","last_change":"2024-09-13T05:00:07.067852+0000","last_active":"2024-09-13T05:00:07.227435+0000","last_peered":"2024-09-13T05:00:07.227435+0000","last_clean":"2024-09-13T05:00:07.227435+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:07.227435+0000","last_undegraded":"2024-09-13T05:00:07.227435+0000","last_fullsized":"2024-09-13T05:00:07.227435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042328299999999999,"stat_sum":{"num_bytes":15830567,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":23,"num_read":6963,"num_read_kb":22566,"num_write":910,"num_write_kb":15653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.2","version":"23'2","reported_seq":120,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093935+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:06.093935+0000","last_peered":"2024-09-13T05:00:06.093935+0000","last_clean":"2024-09-13T05:00:06.093935+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:06.093935+0000","last_undegraded":"2024-09-13T05:00:06.093935+0000","last_fullsized":"2024-09-13T05:00:06.093935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"122'1196","reported_seq":2773,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028836+0000","last_change":"2024-09-13T05:00:06.028836+0000","last_active":"2024-09-13T05:00:06.028836+0000","last_peered":"2024-09-13T05:00:06.028836+0000","last_clean":"2024-09-13T05:00:06.028836+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:06.028836+0000","last_undegraded":"2024-09-13T05:00:06.028836+0000","last_fullsized":"2024-09-13T05:00:06.028836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1196,"log_dups_size":0,"ondisk_log_size":1196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220477,"stat_sum":{"num_bytes":4067826,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":23,"num_read":7005,"num_read_kb":20796,"num_write":1817,"num_write_kb":11791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.056888+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T05:00:06.056888+0000","last_peered":"2024-09-13T05:00:06.056888+0000","last_clean":"2024-09-13T05:00:06.056888+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T05:00:06.056888+0000","last_undegraded":"2024-09-13T05:00:06.056888+0000","last_fullsized":"2024-09-13T05:00:06.056888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"124'441","reported_seq":5059,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.102091+0000","last_change":"2024-09-13T05:00:07.102091+0000","last_active":"2024-09-13T05:00:07.102091+0000","last_peered":"2024-09-13T05:00:07.102091+0000","last_clean":"2024-09-13T05:00:07.102091+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:07.102091+0000","last_undegraded":"2024-09-13T05:00:07.102091+0000","last_fullsized":"2024-09-13T05:00:07.102091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065723400000000003,"stat_sum":{"num_bytes":28878669,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":5759,"num_read_kb":23227,"num_write":423,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093118+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:06.093118+0000","last_peered":"2024-09-13T05:00:06.093118+0000","last_clean":"2024-09-13T05:00:06.093118+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:06.093118+0000","last_undegraded":"2024-09-13T05:00:06.093118+0000","last_fullsized":"2024-09-13T05:00:06.093118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'1026","reported_seq":3238,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.215709+0000","last_change":"2024-09-13T05:00:07.052184+0000","last_active":"2024-09-13T05:00:07.215709+0000","last_peered":"2024-09-13T05:00:07.215709+0000","last_clean":"2024-09-13T05:00:07.215709+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:07.215709+0000","last_undegraded":"2024-09-13T05:00:07.215709+0000","last_fullsized":"2024-09-13T05:00:07.215709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003741753,"stat_sum":{"num_bytes":2686992,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":25,"num_read":6899,"num_read_kb":25000,"num_write":1435,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.3","version":"21'1","reported_seq":119,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.094307+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:06.094307+0000","last_peered":"2024-09-13T05:00:06.094307+0000","last_clean":"2024-09-13T05:00:06.094307+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:06.094307+0000","last_undegraded":"2024-09-13T05:00:06.094307+0000","last_fullsized":"2024-09-13T05:00:06.094307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":201,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T05:00:05.024942+0000","last_change":"2024-09-13T04:59:48.940179+0000","last_active":"2024-09-13T05:00:05.024942+0000","last_peered":"2024-09-13T05:00:05.024942+0000","last_clean":"2024-09-13T05:00:05.024942+0000","last_became_active":"2024-09-13T04:59:48.938034+0000","last_became_peered":"2024-09-13T04:59:48.938034+0000","last_unstale":"2024-09-13T05:00:05.024942+0000","last_undegraded":"2024-09-13T05:00:05.024942+0000","last_fullsized":"2024-09-13T05:00:05.024942+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:51:02.403769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"122'485","reported_seq":3725,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048553+0000","last_change":"2024-09-13T05:00:07.049123+0000","last_active":"2024-09-13T05:00:07.048553+0000","last_peered":"2024-09-13T05:00:07.048553+0000","last_clean":"2024-09-13T05:00:07.048553+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:07.048553+0000","last_undegraded":"2024-09-13T05:00:07.048553+0000","last_fullsized":"2024-09-13T05:00:07.048553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244079999999999,"stat_sum":{"num_bytes":2158608,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":4652,"num_read_kb":13059,"num_write":514,"num_write_kb":10371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.9","version":"124'579","reported_seq":5925,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.257562+0000","last_change":"2024-09-13T05:00:07.087289+0000","last_active":"2024-09-13T05:00:07.257562+0000","last_peered":"2024-09-13T05:00:07.257562+0000","last_clean":"2024-09-13T05:00:07.257562+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:07.257562+0000","last_undegraded":"2024-09-13T05:00:07.257562+0000","last_fullsized":"2024-09-13T05:00:07.257562+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072982600000000004,"stat_sum":{"num_bytes":17874913,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":6863,"num_read_kb":22408,"num_write":702,"num_write_kb":12026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.a","version":"124'462","reported_seq":6936,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.190543+0000","last_change":"2024-09-13T05:00:07.087704+0000","last_active":"2024-09-13T05:00:07.190543+0000","last_peered":"2024-09-13T05:00:07.190543+0000","last_clean":"2024-09-13T05:00:07.190543+0000","last_became_active":"2024-09-13T04:59:48.935630+0000","last_became_peered":"2024-09-13T04:59:48.935630+0000","last_unstale":"2024-09-13T05:00:07.190543+0000","last_undegraded":"2024-09-13T05:00:07.190543+0000","last_fullsized":"2024-09-13T05:00:07.190543+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:27:40.051698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172389999999999,"stat_sum":{"num_bytes":12781457,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":6351,"num_read_kb":24124,"num_write":452,"num_write_kb":14043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":8097453,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.b","version":"124'319","reported_seq":2076,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.197090+0000","last_change":"2024-09-13T05:00:07.057557+0000","last_active":"2024-09-13T05:00:07.197090+0000","last_peered":"2024-09-13T05:00:07.197090+0000","last_clean":"2024-09-13T05:00:07.197090+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T05:00:07.197090+0000","last_undegraded":"2024-09-13T05:00:07.197090+0000","last_fullsized":"2024-09-13T05:00:07.197090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073781599999999997,"stat_sum":{"num_bytes":10190542,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":1663,"num_read_kb":17144,"num_write":300,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.c","version":"124'658","reported_seq":1805,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.186539+0000","last_change":"2024-09-13T05:00:07.068140+0000","last_active":"2024-09-13T05:00:07.186539+0000","last_peered":"2024-09-13T05:00:07.186539+0000","last_clean":"2024-09-13T05:00:07.186539+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:07.186539+0000","last_undegraded":"2024-09-13T05:00:07.186539+0000","last_fullsized":"2024-09-13T05:00:07.186539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050180799999999996,"stat_sum":{"num_bytes":8858126,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":24,"num_read":3102,"num_read_kb":9307,"num_write":850,"num_write_kb":7831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.d","version":"123'435","reported_seq":4594,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087931+0000","last_change":"2024-09-13T05:00:07.088168+0000","last_active":"2024-09-13T05:00:07.087931+0000","last_peered":"2024-09-13T05:00:07.087931+0000","last_clean":"2024-09-13T05:00:07.087931+0000","last_became_active":"2024-09-13T04:59:48.931622+0000","last_became_peered":"2024-09-13T04:59:48.931622+0000","last_unstale":"2024-09-13T05:00:07.087931+0000","last_undegraded":"2024-09-13T05:00:07.087931+0000","last_fullsized":"2024-09-13T05:00:07.087931+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:17:05.641262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001391818,"stat_sum":{"num_bytes":7429674,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":5605,"num_read_kb":35524,"num_write":608,"num_write_kb":15014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":11706466,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.e","version":"122'420","reported_seq":1883,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.042027+0000","last_change":"2024-09-13T05:00:07.042282+0000","last_active":"2024-09-13T05:00:07.042027+0000","last_peered":"2024-09-13T05:00:07.042027+0000","last_clean":"2024-09-13T05:00:07.042027+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:07.042027+0000","last_undegraded":"2024-09-13T05:00:07.042027+0000","last_fullsized":"2024-09-13T05:00:07.042027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057327599999999995,"stat_sum":{"num_bytes":10108580,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":2827,"num_read_kb":9044,"num_write":390,"num_write_kb":5098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.f","version":"124'824","reported_seq":8416,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.254680+0000","last_change":"2024-09-13T05:00:07.068576+0000","last_active":"2024-09-13T05:00:07.254680+0000","last_peered":"2024-09-13T05:00:07.254680+0000","last_clean":"2024-09-13T05:00:07.254680+0000","last_became_active":"2024-09-13T04:59:48.941958+0000","last_became_peered":"2024-09-13T04:59:48.941958+0000","last_unstale":"2024-09-13T05:00:07.254680+0000","last_undegraded":"2024-09-13T05:00:07.254680+0000","last_fullsized":"2024-09-13T05:00:07.254680+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:09:58.793384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071082000000000003,"stat_sum":{"num_bytes":14807155,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":23,"num_read":9762,"num_read_kb":27337,"num_write":1138,"num_write_kb":19486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":7326157,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.10","version":"122'423","reported_seq":7043,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.027979+0000","last_change":"2024-09-13T05:00:06.027979+0000","last_active":"2024-09-13T05:00:06.027979+0000","last_peered":"2024-09-13T05:00:06.027979+0000","last_clean":"2024-09-13T05:00:06.027979+0000","last_became_active":"2024-09-13T04:59:48.932014+0000","last_became_peered":"2024-09-13T04:59:48.932014+0000","last_unstale":"2024-09-13T05:00:06.027979+0000","last_undegraded":"2024-09-13T05:00:06.027979+0000","last_fullsized":"2024-09-13T05:00:06.027979+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:12:17.905901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055876299999999999,"stat_sum":{"num_bytes":13830928,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":32,"num_read":7202,"num_read_kb":16550,"num_write":418,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2709903,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.11","version":"124'392","reported_seq":5074,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.187345+0000","last_change":"2024-09-13T05:00:07.087052+0000","last_active":"2024-09-13T05:00:07.187345+0000","last_peered":"2024-09-13T05:00:07.187345+0000","last_clean":"2024-09-13T05:00:07.187345+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:07.187345+0000","last_undegraded":"2024-09-13T05:00:07.187345+0000","last_fullsized":"2024-09-13T05:00:07.187345+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000684714,"stat_sum":{"num_bytes":22811071,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":5435,"num_read_kb":31446,"num_write":396,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.12","version":"124'432","reported_seq":3586,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.102474+0000","last_change":"2024-09-13T05:00:07.102474+0000","last_active":"2024-09-13T05:00:07.102474+0000","last_peered":"2024-09-13T05:00:07.102474+0000","last_clean":"2024-09-13T05:00:07.102474+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:07.102474+0000","last_undegraded":"2024-09-13T05:00:07.102474+0000","last_fullsized":"2024-09-13T05:00:07.102474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085298500000000001,"stat_sum":{"num_bytes":4570155,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":18,"num_read":3947,"num_read_kb":13255,"num_write":420,"num_write_kb":7303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.13","version":"122'586","reported_seq":1824,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.086833+0000","last_change":"2024-09-13T05:00:07.087037+0000","last_active":"2024-09-13T05:00:07.086833+0000","last_peered":"2024-09-13T05:00:07.086833+0000","last_clean":"2024-09-13T05:00:07.086833+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:07.086833+0000","last_undegraded":"2024-09-13T05:00:07.086833+0000","last_fullsized":"2024-09-13T05:00:07.086833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053792499999999997,"stat_sum":{"num_bytes":8599316,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":3101,"num_read_kb":10798,"num_write":757,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":377872414,"num_objects":1004,"num_object_clones":181,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":656,"num_read":159972,"num_read_kb":651962,"num_write":20866,"num_write_kb":341735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":954,"num_bytes_recovered":70467294,"num_keys_recovered":178,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27303936,"data_stored":248052570,"data_compressed":6026891,"data_compressed_allocated":20779008,"data_compressed_original":243111143,"omap_allocated":0,"internal_metadata":0},"log_size":17543,"ondisk_log_size":17543,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":19544,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43016,"kb_used_data":3704,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94328824,"statfs":{"total":96636764160,"available":96592715776,"internally_reserved":0,"allocated":3792896,"data_stored":22261128,"data_compressed":409733,"data_compressed_allocated":1712128,"data_compressed_original":21094400,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":1.1419999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.92700000000000005}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.054},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.97099999999999997}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":1.1020000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.1859999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30772,"kb_used_data":3748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341068,"statfs":{"total":96636764160,"available":96605253632,"internally_reserved":0,"allocated":3837952,"data_stored":22990142,"data_compressed":314142,"data_compressed_allocated":1712128,"data_compressed_original":21816215,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":0.873},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.96999999999999997}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.05}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.165},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.0129999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.141},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.0820000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53028,"kb_used_data":3864,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318812,"statfs":{"total":96636764160,"available":96582463488,"internally_reserved":0,"allocated":3956736,"data_stored":25404186,"data_compressed":428709,"data_compressed_allocated":1839104,"data_compressed_original":24204698,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.2490000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.0409999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1910000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0860000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58528,"kb_used_data":5268,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94313312,"statfs":{"total":96636764160,"available":96576831488,"internally_reserved":0,"allocated":5394432,"data_stored":38411852,"data_compressed":811081,"data_compressed_allocated":2904064,"data_compressed_original":36913040,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.1539999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84999999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.228},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.012},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.1910000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61472,"kb_used_data":4112,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94310368,"statfs":{"total":96636764160,"available":96573816832,"internally_reserved":0,"allocated":4210688,"data_stored":29755378,"data_compressed":645184,"data_compressed_allocated":2179072,"data_compressed_original":28658130,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.378},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.88}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.66800000000000004}]}]},{"osd":1,"up_from":0,"seq":64424509471,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57388,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94314452,"statfs":{"total":96636764160,"available":96577998848,"internally_reserved":0,"allocated":4227072,"data_stored":28900865,"data_compressed":543997,"data_compressed_allocated":2084864,"data_compressed_original":27691912,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.28399999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.501},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.753}]}]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53576,"kb_used_data":4412,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318264,"statfs":{"total":96636764160,"available":96581902336,"internally_reserved":0,"allocated":4517888,"data_stored":28725013,"data_compressed":559514,"data_compressed_allocated":2174976,"data_compressed_original":27339296,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.89400000000000002}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2752512,"data_stored":28207094,"data_compressed":543997,"data_compressed_allocated":2084864,"data_compressed_original":27691912,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8847360,"data_stored":68621506,"data_compressed":2518998,"data_compressed_allocated":7094272,"data_compressed_original":67349676,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2695168,"data_stored":28471239,"data_compressed":638094,"data_compressed_allocated":2142208,"data_compressed_original":28068306,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3297280,"data_stored":31296936,"data_compressed":634508,"data_compressed_allocated":2359296,"data_compressed_original":30560144,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2482176,"data_stored":24710415,"data_compressed":428709,"data_compressed_allocated":1839104,"data_compressed_original":24204698,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2998272,"data_stored":27447138,"data_compressed":553287,"data_compressed_allocated":2138112,"data_compressed_original":26765856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2359296,"data_stored":22296352,"data_compressed":314142,"data_compressed_allocated":1712128,"data_compressed_original":21816215,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1871872,"data_stored":17001890,"data_compressed":395156,"data_compressed_allocated":1409024,"data_compressed_original":16654336,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:09.804 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T05:00:09.804 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:00:09.804 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:09.872 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:00:09.873 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:09.870+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:09.964 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:09.964 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:09.964 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:09.964 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:09.964 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.927005624, 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.965 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T05:00:09.966 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.967 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.968 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.926959987, 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T05:00:09.969 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.970 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.971 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T05:00:09.972 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.973 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.926911903, 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.974 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.975 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.976 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T05:00:09.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T05:00:09.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.977 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.977 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.977 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T05:00:09.977 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.978 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T05:00:09.979 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T05:00:09.979 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.979 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.979 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.92681582100001, 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.981 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.983 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.984 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.985 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.92676444999999, 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.986 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.987 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.988 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T05:00:09.989 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.926718396, 2024-09-13T05:00:09.990 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T05:00:09.992 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.993 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T05:00:09.994 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.926668335, 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:09.995 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T05:00:09.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T05:00:09.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:09.999 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:10.000 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.809253145, 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.001 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.002 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T05:00:10.003 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T05:00:10.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.005 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T05:00:10.006 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:10.007 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.71780041700001, 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.008 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.009 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T05:00:10.010 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.011 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:10.012 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.71771906399999, 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.013 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.014 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.015 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T05:00:10.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T05:00:10.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.016 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.016 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.016 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T05:00:10.016 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T05:00:10.017 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.717649534, 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.018 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.019 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T05:00:10.020 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T05:00:10.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.022 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.71762225800001, 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.023 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.024 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T05:00:10.025 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.026 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.717523813, 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.028 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.029 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.030 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T05:00:10.031 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.032 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.717454246, 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.033 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:10.034 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.035 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.036 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.037 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.71738377200001, 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.038 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.039 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.040 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T05:00:10.041 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.71735808699999, 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T05:00:10.042 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.043 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:10.044 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.045 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.046 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.717322899, 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.048 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.049 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.050 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T05:00:10.051 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.717298383, 2024-09-13T05:00:10.052 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.053 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.055 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T05:00:10.056 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 108.717229046, 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.057 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:10.058 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.059 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.060 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.061 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:99156 3.0 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 37.564442790000001, 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083885007999999997, 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 99156 2024-09-13T05:00:10.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:10.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396500+0000", 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999933 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404915+0000", 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084155649999999999 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404964+0000", 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8795000000000003e-05 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404987+0000", 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2719999999999999e-05 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405217+0000", 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230163 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405435+0000", 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000218054 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 7", 2024-09-13T05:00:10.066 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.406287+0000", 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00085163700000000003 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.470407+0000", 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.064120636999999994 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480294+0000", 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098868949999999997 2024-09-13T05:00:10.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480345+0000", 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1116999999999999e-05 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480391+0000", 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6224999999999999e-05 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.068 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.069 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:10.069 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:10.069 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:10.070 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:10.070 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:10.071 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:10.071 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:10.072 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:00:10.072 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:10.065+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:10.072 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T05:00:10.073 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:10.073 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:10.083 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:10.083 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:10.102 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:10.123 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:10.133 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:10.134 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:10.145 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:10.160 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-13T05:00:10.047688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":378462801,"num_objects":1008,"num_object_clones":181,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1008,"num_whiteouts":656,"num_read":160088,"num_read_kb":652062,"num_write":21027,"num_write_kb":344153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":956,"num_bytes_recovered":71057662,"num_keys_recovered":178,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17663,"ondisk_log_size":17663,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":357780,"kb_used_data":29236,"kb_used_omap":10,"kb_used_meta":328437,"kb_avail":660245100,"statfs":{"total":676457349120,"available":676090982400,"internally_reserved":0,"allocated":29937664,"data_stored":196448564,"data_compressed":3712360,"data_compressed_allocated":14606336,"data_compressed_original":187717691,"omap_allocated":11149,"internal_metadata":336319603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-149127859,"num_objects":-400,"num_object_clones":-158,"num_object_copies":-800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-400,"num_whiteouts":-153,"num_read":1410,"num_read_kb":1206,"num_write":273,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024249"},"pg_stats":[{"pgid":"3.1f","version":"123'552","reported_seq":3411,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.207217+0000","last_change":"2024-09-13T05:00:06.029272+0000","last_active":"2024-09-13T05:00:06.207217+0000","last_peered":"2024-09-13T05:00:06.207217+0000","last_clean":"2024-09-13T05:00:06.207217+0000","last_became_active":"2024-09-13T04:59:49.939066+0000","last_became_peered":"2024-09-13T04:59:49.939066+0000","last_unstale":"2024-09-13T05:00:06.207217+0000","last_undegraded":"2024-09-13T05:00:06.207217+0000","last_fullsized":"2024-09-13T05:00:06.207217+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:55:39.536667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014398939999999999,"stat_sum":{"num_bytes":26868257,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":23,"num_read":4215,"num_read_kb":16134,"num_write":573,"num_write_kb":9809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":7347311,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.1e","version":"123'377","reported_seq":2166,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.159581+0000","last_change":"2024-09-13T05:00:06.029915+0000","last_active":"2024-09-13T05:00:06.159581+0000","last_peered":"2024-09-13T05:00:06.159581+0000","last_clean":"2024-09-13T05:00:06.159581+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:06.159581+0000","last_undegraded":"2024-09-13T05:00:06.159581+0000","last_fullsized":"2024-09-13T05:00:06.159581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013133890000000001,"stat_sum":{"num_bytes":3604325,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":2755,"num_read_kb":18337,"num_write":333,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.1d","version":"124'519","reported_seq":3708,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087850+0000","last_change":"2024-09-13T05:00:07.087850+0000","last_active":"2024-09-13T05:00:07.087850+0000","last_peered":"2024-09-13T05:00:07.087850+0000","last_clean":"2024-09-13T05:00:07.087850+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:07.087850+0000","last_undegraded":"2024-09-13T05:00:07.087850+0000","last_fullsized":"2024-09-13T05:00:07.087850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001160021,"stat_sum":{"num_bytes":20336804,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":4168,"num_read_kb":23676,"num_write":657,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1c","version":"124'578","reported_seq":1861,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.114431+0000","last_change":"2024-09-13T05:00:07.068194+0000","last_active":"2024-09-13T05:00:07.114431+0000","last_peered":"2024-09-13T05:00:07.114431+0000","last_clean":"2024-09-13T05:00:07.114431+0000","last_became_active":"2024-09-13T04:59:48.932372+0000","last_became_peered":"2024-09-13T04:59:48.932372+0000","last_unstale":"2024-09-13T05:00:07.114431+0000","last_undegraded":"2024-09-13T05:00:07.114431+0000","last_fullsized":"2024-09-13T05:00:07.114431+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:06:24.843964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064349000000000003,"stat_sum":{"num_bytes":7454720,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":2097,"num_read_kb":8182,"num_write":715,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":5244301,"num_keys_recovered":62,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1b","version":"122'500","reported_seq":5015,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048707+0000","last_change":"2024-09-13T05:00:07.049219+0000","last_active":"2024-09-13T05:00:07.048707+0000","last_peered":"2024-09-13T05:00:07.048707+0000","last_clean":"2024-09-13T05:00:07.048707+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:07.048707+0000","last_undegraded":"2024-09-13T05:00:07.048707+0000","last_fullsized":"2024-09-13T05:00:07.048707+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001125708,"stat_sum":{"num_bytes":12918784,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":5625,"num_read_kb":17585,"num_write":555,"num_write_kb":6766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.1a","version":"122'480","reported_seq":3818,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.191350+0000","last_change":"2024-09-13T05:00:06.028415+0000","last_active":"2024-09-13T05:00:06.191350+0000","last_peered":"2024-09-13T05:00:06.191350+0000","last_clean":"2024-09-13T05:00:06.191350+0000","last_became_active":"2024-09-13T04:59:48.942009+0000","last_became_peered":"2024-09-13T04:59:48.942009+0000","last_unstale":"2024-09-13T05:00:06.191350+0000","last_undegraded":"2024-09-13T05:00:06.191350+0000","last_fullsized":"2024-09-13T05:00:06.191350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:38:34.898545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089277099999999999,"stat_sum":{"num_bytes":17659856,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":21,"num_read":4697,"num_read_kb":13741,"num_write":538,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":6543600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.19","version":"124'434","reported_seq":18907,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.049966+0000","last_change":"2024-09-13T05:00:07.049966+0000","last_active":"2024-09-13T05:00:07.049966+0000","last_peered":"2024-09-13T05:00:07.049966+0000","last_clean":"2024-09-13T05:00:07.049966+0000","last_became_active":"2024-09-13T04:59:48.941448+0000","last_became_peered":"2024-09-13T04:59:48.941448+0000","last_unstale":"2024-09-13T05:00:07.049966+0000","last_undegraded":"2024-09-13T05:00:07.049966+0000","last_fullsized":"2024-09-13T05:00:07.049966+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:02:00.880300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016055209999999999,"stat_sum":{"num_bytes":14492081,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":19059,"num_read_kb":95954,"num_write":479,"num_write_kb":19269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":16500093,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.18","version":"122'467","reported_seq":2965,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028609+0000","last_change":"2024-09-13T05:00:06.029147+0000","last_active":"2024-09-13T05:00:06.028609+0000","last_peered":"2024-09-13T05:00:06.028609+0000","last_clean":"2024-09-13T05:00:06.028609+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:06.028609+0000","last_undegraded":"2024-09-13T05:00:06.028609+0000","last_fullsized":"2024-09-13T05:00:06.028609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014909400000000001,"stat_sum":{"num_bytes":17188355,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":20,"num_read":3251,"num_read_kb":9829,"num_write":549,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.17","version":"121'312","reported_seq":3663,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.029225+0000","last_change":"2024-09-13T05:00:06.029482+0000","last_active":"2024-09-13T05:00:06.029225+0000","last_peered":"2024-09-13T05:00:06.029225+0000","last_clean":"2024-09-13T05:00:06.029225+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:06.029225+0000","last_undegraded":"2024-09-13T05:00:06.029225+0000","last_fullsized":"2024-09-13T05:00:06.029225+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001461216,"stat_sum":{"num_bytes":14360724,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":3215,"num_read_kb":21670,"num_write":303,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.16","version":"122'529","reported_seq":2504,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.041654+0000","last_change":"2024-09-13T05:00:07.042229+0000","last_active":"2024-09-13T05:00:07.041654+0000","last_peered":"2024-09-13T05:00:07.041654+0000","last_clean":"2024-09-13T05:00:07.041654+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:07.041654+0000","last_undegraded":"2024-09-13T05:00:07.041654+0000","last_fullsized":"2024-09-13T05:00:07.041654+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064906100000000004,"stat_sum":{"num_bytes":6348907,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":3479,"num_read_kb":19817,"num_write":649,"num_write_kb":7274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.15","version":"122'406","reported_seq":1162,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.068221+0000","last_change":"2024-09-13T05:00:07.068378+0000","last_active":"2024-09-13T05:00:07.068221+0000","last_peered":"2024-09-13T05:00:07.068221+0000","last_clean":"2024-09-13T05:00:07.068221+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:07.068221+0000","last_undegraded":"2024-09-13T05:00:07.068221+0000","last_fullsized":"2024-09-13T05:00:07.068221+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063102299999999998,"stat_sum":{"num_bytes":10107399,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1530,"num_read_kb":10660,"num_write":446,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.14","version":"124'698","reported_seq":4204,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087387+0000","last_change":"2024-09-13T05:00:07.087387+0000","last_active":"2024-09-13T05:00:07.087387+0000","last_peered":"2024-09-13T05:00:07.087387+0000","last_clean":"2024-09-13T05:00:07.087387+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:07.087387+0000","last_undegraded":"2024-09-13T05:00:07.087387+0000","last_fullsized":"2024-09-13T05:00:07.087387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083436199999999995,"stat_sum":{"num_bytes":7195972,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":5421,"num_read_kb":13519,"num_write":911,"num_write_kb":5068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":118,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.086491+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:07.086491+0000","last_peered":"2024-09-13T05:00:07.086491+0000","last_clean":"2024-09-13T05:00:07.086491+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:07.086491+0000","last_undegraded":"2024-09-13T05:00:07.086491+0000","last_fullsized":"2024-09-13T05:00:07.086491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"124'308","reported_seq":2566,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.187614+0000","last_change":"2024-09-13T05:00:07.103142+0000","last_active":"2024-09-13T05:00:07.187614+0000","last_peered":"2024-09-13T05:00:07.187614+0000","last_clean":"2024-09-13T05:00:07.187614+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:07.187614+0000","last_undegraded":"2024-09-13T05:00:07.187614+0000","last_fullsized":"2024-09-13T05:00:07.187614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001116515,"stat_sum":{"num_bytes":18004187,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":23,"num_read":2154,"num_read_kb":23093,"num_write":280,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048249+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:07.048249+0000","last_peered":"2024-09-13T05:00:07.048249+0000","last_clean":"2024-09-13T05:00:07.048249+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:07.048249+0000","last_undegraded":"2024-09-13T05:00:07.048249+0000","last_fullsized":"2024-09-13T05:00:07.048249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"124'715","reported_seq":2032,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.049955+0000","last_change":"2024-09-13T05:00:07.049955+0000","last_active":"2024-09-13T05:00:07.049955+0000","last_peered":"2024-09-13T05:00:07.049955+0000","last_clean":"2024-09-13T05:00:07.049955+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:07.049955+0000","last_undegraded":"2024-09-13T05:00:07.049955+0000","last_fullsized":"2024-09-13T05:00:07.049955+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016657849999999999,"stat_sum":{"num_bytes":6287243,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":17,"num_read":4240,"num_read_kb":12408,"num_write":980,"num_write_kb":8392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093521+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:06.093521+0000","last_peered":"2024-09-13T05:00:06.093521+0000","last_clean":"2024-09-13T05:00:06.093521+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:06.093521+0000","last_undegraded":"2024-09-13T05:00:06.093521+0000","last_fullsized":"2024-09-13T05:00:06.093521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"122'621","reported_seq":1698,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028145+0000","last_change":"2024-09-13T05:00:06.028461+0000","last_active":"2024-09-13T05:00:06.028145+0000","last_peered":"2024-09-13T05:00:06.028145+0000","last_clean":"2024-09-13T05:00:06.028145+0000","last_became_active":"2024-09-13T04:59:29.177938+0000","last_became_peered":"2024-09-13T04:59:29.177938+0000","last_unstale":"2024-09-13T05:00:06.028145+0000","last_undegraded":"2024-09-13T05:00:06.028145+0000","last_fullsized":"2024-09-13T05:00:06.028145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:05:09.883322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085147300000000005,"stat_sum":{"num_bytes":6578176,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":29,"num_read":3117,"num_read_kb":14959,"num_write":796,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":117,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T05:00:05.024790+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:05.024790+0000","last_peered":"2024-09-13T05:00:05.024790+0000","last_clean":"2024-09-13T05:00:05.024790+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:05.024790+0000","last_undegraded":"2024-09-13T05:00:05.024790+0000","last_fullsized":"2024-09-13T05:00:05.024790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"122'536","reported_seq":2332,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.041416+0000","last_change":"2024-09-13T05:00:07.041865+0000","last_active":"2024-09-13T05:00:07.041416+0000","last_peered":"2024-09-13T05:00:07.041416+0000","last_clean":"2024-09-13T05:00:07.041416+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:07.041416+0000","last_undegraded":"2024-09-13T05:00:07.041416+0000","last_fullsized":"2024-09-13T05:00:07.041416+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055185599999999998,"stat_sum":{"num_bytes":2982045,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":3813,"num_read_kb":10813,"num_write":572,"num_write_kb":6332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.3","version":"124'833","reported_seq":4650,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.227435+0000","last_change":"2024-09-13T05:00:07.067852+0000","last_active":"2024-09-13T05:00:07.227435+0000","last_peered":"2024-09-13T05:00:07.227435+0000","last_clean":"2024-09-13T05:00:07.227435+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:07.227435+0000","last_undegraded":"2024-09-13T05:00:07.227435+0000","last_fullsized":"2024-09-13T05:00:07.227435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042328299999999999,"stat_sum":{"num_bytes":15830567,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":23,"num_read":6963,"num_read_kb":22566,"num_write":910,"num_write_kb":15653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.2","version":"23'2","reported_seq":120,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093935+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:06.093935+0000","last_peered":"2024-09-13T05:00:06.093935+0000","last_clean":"2024-09-13T05:00:06.093935+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:06.093935+0000","last_undegraded":"2024-09-13T05:00:06.093935+0000","last_fullsized":"2024-09-13T05:00:06.093935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"122'1196","reported_seq":2773,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.028836+0000","last_change":"2024-09-13T05:00:06.028836+0000","last_active":"2024-09-13T05:00:06.028836+0000","last_peered":"2024-09-13T05:00:06.028836+0000","last_clean":"2024-09-13T05:00:06.028836+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:06.028836+0000","last_undegraded":"2024-09-13T05:00:06.028836+0000","last_fullsized":"2024-09-13T05:00:06.028836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1196,"log_dups_size":0,"ondisk_log_size":1196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220477,"stat_sum":{"num_bytes":4067826,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":23,"num_read":7005,"num_read_kb":20796,"num_write":1817,"num_write_kb":11791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.056888+0000","last_change":"2024-09-13T04:57:56.500205+0000","last_active":"2024-09-13T05:00:06.056888+0000","last_peered":"2024-09-13T05:00:06.056888+0000","last_clean":"2024-09-13T05:00:06.056888+0000","last_became_active":"2024-09-13T04:57:54.160823+0000","last_became_peered":"2024-09-13T04:57:54.160823+0000","last_unstale":"2024-09-13T05:00:06.056888+0000","last_undegraded":"2024-09-13T05:00:06.056888+0000","last_fullsized":"2024-09-13T05:00:06.056888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:10:36.313878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075892699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"124'441","reported_seq":5059,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.102091+0000","last_change":"2024-09-13T05:00:07.102091+0000","last_active":"2024-09-13T05:00:07.102091+0000","last_peered":"2024-09-13T05:00:07.102091+0000","last_clean":"2024-09-13T05:00:07.102091+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:07.102091+0000","last_undegraded":"2024-09-13T05:00:07.102091+0000","last_fullsized":"2024-09-13T05:00:07.102091+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065723400000000003,"stat_sum":{"num_bytes":28878669,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":5759,"num_read_kb":23227,"num_write":423,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.093118+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:06.093118+0000","last_peered":"2024-09-13T05:00:06.093118+0000","last_clean":"2024-09-13T05:00:06.093118+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:06.093118+0000","last_undegraded":"2024-09-13T05:00:06.093118+0000","last_fullsized":"2024-09-13T05:00:06.093118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'1026","reported_seq":3238,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.215709+0000","last_change":"2024-09-13T05:00:07.052184+0000","last_active":"2024-09-13T05:00:07.215709+0000","last_peered":"2024-09-13T05:00:07.215709+0000","last_clean":"2024-09-13T05:00:07.215709+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:07.215709+0000","last_undegraded":"2024-09-13T05:00:07.215709+0000","last_fullsized":"2024-09-13T05:00:07.215709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003741753,"stat_sum":{"num_bytes":2686992,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":25,"num_read":6899,"num_read_kb":25000,"num_write":1435,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"2.3","version":"21'1","reported_seq":119,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.094307+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:06.094307+0000","last_peered":"2024-09-13T05:00:06.094307+0000","last_clean":"2024-09-13T05:00:06.094307+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:06.094307+0000","last_undegraded":"2024-09-13T05:00:06.094307+0000","last_fullsized":"2024-09-13T05:00:06.094307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":201,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T05:00:05.024942+0000","last_change":"2024-09-13T04:59:48.940179+0000","last_active":"2024-09-13T05:00:05.024942+0000","last_peered":"2024-09-13T05:00:05.024942+0000","last_clean":"2024-09-13T05:00:05.024942+0000","last_became_active":"2024-09-13T04:59:48.938034+0000","last_became_peered":"2024-09-13T04:59:48.938034+0000","last_unstale":"2024-09-13T05:00:05.024942+0000","last_undegraded":"2024-09-13T05:00:05.024942+0000","last_fullsized":"2024-09-13T05:00:05.024942+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:51:02.403769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"122'485","reported_seq":3725,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.048553+0000","last_change":"2024-09-13T05:00:07.049123+0000","last_active":"2024-09-13T05:00:07.048553+0000","last_peered":"2024-09-13T05:00:07.048553+0000","last_clean":"2024-09-13T05:00:07.048553+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:07.048553+0000","last_undegraded":"2024-09-13T05:00:07.048553+0000","last_fullsized":"2024-09-13T05:00:07.048553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244079999999999,"stat_sum":{"num_bytes":2158608,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":4652,"num_read_kb":13059,"num_write":514,"num_write_kb":10371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.9","version":"124'579","reported_seq":5925,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.257562+0000","last_change":"2024-09-13T05:00:07.087289+0000","last_active":"2024-09-13T05:00:07.257562+0000","last_peered":"2024-09-13T05:00:07.257562+0000","last_clean":"2024-09-13T05:00:07.257562+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:07.257562+0000","last_undegraded":"2024-09-13T05:00:07.257562+0000","last_fullsized":"2024-09-13T05:00:07.257562+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072982600000000004,"stat_sum":{"num_bytes":17874913,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":6863,"num_read_kb":22408,"num_write":702,"num_write_kb":12026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.a","version":"124'462","reported_seq":6936,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.190543+0000","last_change":"2024-09-13T05:00:07.087704+0000","last_active":"2024-09-13T05:00:07.190543+0000","last_peered":"2024-09-13T05:00:07.190543+0000","last_clean":"2024-09-13T05:00:07.190543+0000","last_became_active":"2024-09-13T04:59:48.935630+0000","last_became_peered":"2024-09-13T04:59:48.935630+0000","last_unstale":"2024-09-13T05:00:07.190543+0000","last_undegraded":"2024-09-13T05:00:07.190543+0000","last_fullsized":"2024-09-13T05:00:07.190543+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:27:40.051698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011172389999999999,"stat_sum":{"num_bytes":12781457,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":6351,"num_read_kb":24124,"num_write":452,"num_write_kb":14043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":8097453,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.b","version":"124'319","reported_seq":2076,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.197090+0000","last_change":"2024-09-13T05:00:07.057557+0000","last_active":"2024-09-13T05:00:07.197090+0000","last_peered":"2024-09-13T05:00:07.197090+0000","last_clean":"2024-09-13T05:00:07.197090+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T05:00:07.197090+0000","last_undegraded":"2024-09-13T05:00:07.197090+0000","last_fullsized":"2024-09-13T05:00:07.197090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:47:59.151239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073781599999999997,"stat_sum":{"num_bytes":10190542,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":1663,"num_read_kb":17144,"num_write":300,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.c","version":"124'658","reported_seq":1805,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.186539+0000","last_change":"2024-09-13T05:00:07.068140+0000","last_active":"2024-09-13T05:00:07.186539+0000","last_peered":"2024-09-13T05:00:07.186539+0000","last_clean":"2024-09-13T05:00:07.186539+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:07.186539+0000","last_undegraded":"2024-09-13T05:00:07.186539+0000","last_fullsized":"2024-09-13T05:00:07.186539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050180799999999996,"stat_sum":{"num_bytes":8858126,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":24,"num_read":3102,"num_read_kb":9307,"num_write":850,"num_write_kb":7831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.d","version":"123'435","reported_seq":4594,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.087931+0000","last_change":"2024-09-13T05:00:07.088168+0000","last_active":"2024-09-13T05:00:07.087931+0000","last_peered":"2024-09-13T05:00:07.087931+0000","last_clean":"2024-09-13T05:00:07.087931+0000","last_became_active":"2024-09-13T04:59:48.931622+0000","last_became_peered":"2024-09-13T04:59:48.931622+0000","last_unstale":"2024-09-13T05:00:07.087931+0000","last_undegraded":"2024-09-13T05:00:07.087931+0000","last_fullsized":"2024-09-13T05:00:07.087931+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:17:05.641262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001391818,"stat_sum":{"num_bytes":7429674,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":5605,"num_read_kb":35524,"num_write":608,"num_write_kb":15014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":11706466,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.e","version":"122'420","reported_seq":1883,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.042027+0000","last_change":"2024-09-13T05:00:07.042282+0000","last_active":"2024-09-13T05:00:07.042027+0000","last_peered":"2024-09-13T05:00:07.042027+0000","last_clean":"2024-09-13T05:00:07.042027+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:07.042027+0000","last_undegraded":"2024-09-13T05:00:07.042027+0000","last_fullsized":"2024-09-13T05:00:07.042027+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057327599999999995,"stat_sum":{"num_bytes":10108580,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":2827,"num_read_kb":9044,"num_write":390,"num_write_kb":5098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.f","version":"124'824","reported_seq":8416,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.254680+0000","last_change":"2024-09-13T05:00:07.068576+0000","last_active":"2024-09-13T05:00:07.254680+0000","last_peered":"2024-09-13T05:00:07.254680+0000","last_clean":"2024-09-13T05:00:07.254680+0000","last_became_active":"2024-09-13T04:59:48.941958+0000","last_became_peered":"2024-09-13T04:59:48.941958+0000","last_unstale":"2024-09-13T05:00:07.254680+0000","last_undegraded":"2024-09-13T05:00:07.254680+0000","last_fullsized":"2024-09-13T05:00:07.254680+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:09:58.793384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071082000000000003,"stat_sum":{"num_bytes":14807155,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":23,"num_read":9762,"num_read_kb":27337,"num_write":1138,"num_write_kb":19486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":7326157,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.10","version":"122'423","reported_seq":7043,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-13T05:00:06.027979+0000","last_change":"2024-09-13T05:00:06.027979+0000","last_active":"2024-09-13T05:00:06.027979+0000","last_peered":"2024-09-13T05:00:06.027979+0000","last_clean":"2024-09-13T05:00:06.027979+0000","last_became_active":"2024-09-13T04:59:48.932014+0000","last_became_peered":"2024-09-13T04:59:48.932014+0000","last_unstale":"2024-09-13T05:00:06.027979+0000","last_undegraded":"2024-09-13T05:00:06.027979+0000","last_fullsized":"2024-09-13T05:00:06.027979+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:12:17.905901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055876299999999999,"stat_sum":{"num_bytes":13830928,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":32,"num_read":7202,"num_read_kb":16550,"num_write":418,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2709903,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"9"}]},{"pgid":"3.11","version":"124'392","reported_seq":5074,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.187345+0000","last_change":"2024-09-13T05:00:07.087052+0000","last_active":"2024-09-13T05:00:07.187345+0000","last_peered":"2024-09-13T05:00:07.187345+0000","last_clean":"2024-09-13T05:00:07.187345+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:07.187345+0000","last_undegraded":"2024-09-13T05:00:07.187345+0000","last_fullsized":"2024-09-13T05:00:07.187345+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000684714,"stat_sum":{"num_bytes":22811071,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":22,"num_read":5435,"num_read_kb":31446,"num_write":396,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.12","version":"124'432","reported_seq":3586,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.102474+0000","last_change":"2024-09-13T05:00:07.102474+0000","last_active":"2024-09-13T05:00:07.102474+0000","last_peered":"2024-09-13T05:00:07.102474+0000","last_clean":"2024-09-13T05:00:07.102474+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:07.102474+0000","last_undegraded":"2024-09-13T05:00:07.102474+0000","last_fullsized":"2024-09-13T05:00:07.102474+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085298500000000001,"stat_sum":{"num_bytes":4570155,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":18,"num_read":3947,"num_read_kb":13255,"num_write":420,"num_write_kb":7303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"a"}]},{"pgid":"3.13","version":"122'586","reported_seq":1824,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:00:07.086833+0000","last_change":"2024-09-13T05:00:07.087037+0000","last_active":"2024-09-13T05:00:07.086833+0000","last_peered":"2024-09-13T05:00:07.086833+0000","last_clean":"2024-09-13T05:00:07.086833+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:07.086833+0000","last_undegraded":"2024-09-13T05:00:07.086833+0000","last_fullsized":"2024-09-13T05:00:07.086833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053792499999999997,"stat_sum":{"num_bytes":8599316,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":3101,"num_read_kb":10798,"num_write":757,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":377872414,"num_objects":1004,"num_object_clones":181,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":656,"num_read":159972,"num_read_kb":651962,"num_write":20866,"num_write_kb":341735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":954,"num_bytes_recovered":70467294,"num_keys_recovered":178,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27303936,"data_stored":248052570,"data_compressed":6026891,"data_compressed_allocated":20779008,"data_compressed_original":243111143,"omap_allocated":0,"internal_metadata":0},"log_size":17543,"ondisk_log_size":17543,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":19544,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43016,"kb_used_data":3704,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94328824,"statfs":{"total":96636764160,"available":96592715776,"internally_reserved":0,"allocated":3792896,"data_stored":22261128,"data_compressed":409733,"data_compressed_allocated":1712128,"data_compressed_original":21094400,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":1.1419999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.92700000000000005}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.054},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.97099999999999997}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":1.1020000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.1859999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30772,"kb_used_data":3748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341068,"statfs":{"total":96636764160,"available":96605253632,"internally_reserved":0,"allocated":3837952,"data_stored":22990142,"data_compressed":314142,"data_compressed_allocated":1712128,"data_compressed_original":21816215,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":0.873},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.96999999999999997}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.05}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.165},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.0129999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.141},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.0820000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53028,"kb_used_data":3864,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318812,"statfs":{"total":96636764160,"available":96582463488,"internally_reserved":0,"allocated":3956736,"data_stored":25404186,"data_compressed":428709,"data_compressed_allocated":1839104,"data_compressed_original":24204698,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.2490000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.0409999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1910000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0860000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58528,"kb_used_data":5268,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94313312,"statfs":{"total":96636764160,"available":96576831488,"internally_reserved":0,"allocated":5394432,"data_stored":38411852,"data_compressed":811081,"data_compressed_allocated":2904064,"data_compressed_original":36913040,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.1539999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84999999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.228},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.0940000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.012},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.1910000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61472,"kb_used_data":4112,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94310368,"statfs":{"total":96636764160,"available":96573816832,"internally_reserved":0,"allocated":4210688,"data_stored":29755378,"data_compressed":645184,"data_compressed_allocated":2179072,"data_compressed_original":28658130,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.378},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.88}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.66800000000000004}]}]},{"osd":1,"up_from":0,"seq":64424509471,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57388,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94314452,"statfs":{"total":96636764160,"available":96577998848,"internally_reserved":0,"allocated":4227072,"data_stored":28900865,"data_compressed":543997,"data_compressed_allocated":2084864,"data_compressed_original":27691912,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.28399999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.501},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.753}]}]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53576,"kb_used_data":4412,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94318264,"statfs":{"total":96636764160,"available":96581902336,"internally_reserved":0,"allocated":4517888,"data_stored":28725013,"data_compressed":559514,"data_compressed_allocated":2174976,"data_compressed_original":27339296,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.89400000000000002}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2752512,"data_stored":28207094,"data_compressed":543997,"data_compressed_allocated":2084864,"data_compressed_original":27691912,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8847360,"data_stored":68621506,"data_compressed":2518998,"data_compressed_allocated":7094272,"data_compressed_original":67349676,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2695168,"data_stored":28471239,"data_compressed":638094,"data_compressed_allocated":2142208,"data_compressed_original":28068306,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3297280,"data_stored":31296936,"data_compressed":634508,"data_compressed_allocated":2359296,"data_compressed_original":30560144,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2482176,"data_stored":24710415,"data_compressed":428709,"data_compressed_allocated":1839104,"data_compressed_original":24204698,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2998272,"data_stored":27447138,"data_compressed":553287,"data_compressed_allocated":2138112,"data_compressed_original":26765856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2359296,"data_stored":22296352,"data_compressed":314142,"data_compressed_allocated":1712128,"data_compressed_original":21816215,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1871872,"data_stored":17001890,"data_compressed":395156,"data_compressed_allocated":1409024,"data_compressed_original":16654336,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:10.169 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:10.169 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:10.171 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:10.165+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:10.242 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:10.243 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:10.258 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:00:10.301 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:10.297+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:10.373 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:10.369+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:10.420 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:10.421 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:10.435 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:00:10.474 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:10.470+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:10.575 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:10.574+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:10.598 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:10.598 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:10.613 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:00:10.675 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:10.669+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:10.737 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:10.737 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:10.751 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:00:10.776 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:10.770+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:10.877 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:10.874+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:10.912 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:10.912 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:10.927 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:00:10.978 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:10.973+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:11.050 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:11.050 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:11.064 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:00:11.110 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:11.106+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:11.179 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:11.173+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:11.214 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:11.214 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:11.230 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:00:11.280 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:11.273+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:11.381 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:11.377+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:11.396 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:11.396 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:11.412 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:00:11.482 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:11.478+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:11.542 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:11.542 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:11.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:00:11.583 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:11.577+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:11.685 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:11.681+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:11.723 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:11.723 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:11.738 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:00:11.786 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:11.781+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:11.858 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:11.858 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:11.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:00:11.918 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:11.914+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:11.987 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:11.981+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:11.993 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:11.993 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:12.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:00:12.084 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:00:12.097 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:12.094+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:12.182 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:12.182 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:12.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:00:12.189 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:12.185+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:12.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:00:12.290 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:12.285+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:12.342 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:12.342 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:12.359 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:12.359 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:12.373 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:12.374 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:12.377 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:12.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:00:12.390 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:12.385+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:12.394 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:12.394 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:12.415 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:12.415 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:12.435 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:12.435 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:12.455 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:12.476 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:12.477 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:12.491 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:12.485+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:18956 3.11 e35/24 3:8c1e2dec:::rbd_data.112ab86ccaf9.0000000000000001:head v 35'55, mlcod=35'52)", 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.046934632, 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045656874, 2024-09-13T05:00:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461159+0000", 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.461157+0000", 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465784+0000", 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046268300000000002 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465810+0000", 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5448000000000001e-05 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465819+0000", 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7359999999999996e-06 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465876+0000", 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6317999999999998e-05 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.465905+0000", 2024-09-13T05:00:12.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9394e-05 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506784+0000", 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040879036000000001 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.506816+0000", 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1687999999999999e-05 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103023 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 468992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.713405846999997, 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042297819, 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103023 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794688+0000", 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794689+0000", 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7060000000000001e-06 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794693+0000", 2024-09-13T05:00:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.811e-06 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794696+0000", 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1429999999999998e-06 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836839+0000", 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042142577000000001 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836920+0000", 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1012000000000003e-05 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.836986+0000", 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5841999999999997e-05 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103024 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 482304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.713394979999997, 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042470763000000002, 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103024 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794699+0000", 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794698+0000", 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794700+0000", 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.567e-06 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794704+0000", 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8399999999999997e-06 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794707+0000", 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1209999999999998e-06 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837011+0000", 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042303828000000002 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837096+0000", 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5148999999999994e-05 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837169+0000", 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3521000000000003e-05 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.522 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103025 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.713384148000003, 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042586098000000003, 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103025 2024-09-13T05:00:12.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794710+0000", 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794709+0000", 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794711+0000", 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5999999999999999e-06 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794715+0000", 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8260000000000003e-06 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794718+0000", 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837196+0000", 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042477695000000003 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837252+0000", 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6808000000000002e-05 2024-09-13T05:00:12.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837296+0000", 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3327e-05 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103026 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 508928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.713373470999997, 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042722826999999998, 2024-09-13T05:00:12.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103026 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794720+0000", 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794721+0000", 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4619999999999999e-06 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794725+0000", 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7890000000000001e-06 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794736+0000", 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1202999999999999e-05 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837322+0000", 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042585852 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837386+0000", 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3942000000000005e-05 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837443+0000", 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6841e-05 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103027 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.713354168000002, 2024-09-13T05:00:12.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042880907000000003, 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103027 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794740+0000", 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794739+0000", 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794741+0000", 2024-09-13T05:00:12.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8279999999999999e-06 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794745+0000", 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0589999999999996e-06 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.794748+0000", 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837472+0000", 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.04272372 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837547+0000", 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4794000000000003e-05 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.837620+0000", 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3646999999999995e-05 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.536 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103275 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 414720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706312922999999, 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042255839000000003, 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103275 2024-09-13T05:00:12.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801781+0000", 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801780+0000", 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801783+0000", 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.757e-06 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801791+0000", 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6830000000000008e-06 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801799+0000", 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.157e-06 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843846+0000", 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042047227999999999 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.843943+0000", 2024-09-13T05:00:12.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7310000000000002e-05 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844037+0000", 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3160999999999998e-05 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103277 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 428032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.541 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706289427000002, 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042336314, 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103277 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801804+0000", 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801807+0000", 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8090000000000001e-06 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801814+0000", 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6669999999999996e-06 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801820+0000", 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0419999999999999e-06 2024-09-13T05:00:12.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844063+0000", 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042242793000000001 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844103+0000", 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9526000000000001e-05 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844141+0000", 2024-09-13T05:00:12.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7892999999999998e-05 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103278 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 441344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706268158999997, 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042426288, 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103278 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801826+0000", 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801825+0000", 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801828+0000", 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.525e-06 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801835+0000", 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8669999999999994e-06 2024-09-13T05:00:12.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801841+0000", 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.677e-06 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844165+0000", 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042323652000000003 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844210+0000", 2024-09-13T05:00:12.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5623e-05 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844252+0000", 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1384999999999998e-05 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103279 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 454656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:12.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706247150999999, 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042502914000000003, 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103279 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801847+0000", 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801846+0000", 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801849+0000", 2024-09-13T05:00:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801858+0000", 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9649999999999997e-06 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801864+0000", 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3119999999999999e-06 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844274+0000", 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042409413999999999 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844313+0000", 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9131999999999997e-05 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844349+0000", 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6751000000000001e-05 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103280 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 470016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706224282000001, 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042583929, 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103280 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801869+0000", 2024-09-13T05:00:12.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801872+0000", 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801880+0000", 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0800000000000006e-06 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801886+0000", 2024-09-13T05:00:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2339999999999999e-06 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844372+0000", 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042486084 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844413+0000", 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0448999999999998e-05 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844453+0000", 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0550999999999999e-05 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103281 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706188965000003, 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042747949, 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103281 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801905+0000", 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801904+0000", 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801907+0000", 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1690000000000001e-06 2024-09-13T05:00:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801920+0000", 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2082000000000001e-05 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801926+0000", 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7360000000000004e-06 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844484+0000", 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042557709999999999 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844552+0000", 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8396e-05 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844653+0000", 2024-09-13T05:00:12.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100295 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103282 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 496640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706160001000001, 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042899236, 2024-09-13T05:00:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103282 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801934+0000", 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801933+0000", 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801961+0000", 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7827999999999999e-05 2024-09-13T05:00:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801971+0000", 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0225e-05 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801981+0000", 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9280000000000005e-06 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844692+0000", 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042711256000000003 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844769+0000", 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6375e-05 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844833+0000", 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4071000000000004e-05 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103283 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706105909999998, 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025461000000001, 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103283 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801988+0000", 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801987+0000", 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801990+0000", 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2969999999999999e-06 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.801999+0000", 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8470000000000002e-06 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802006+0000", 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2169999999999999e-06 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844871+0000", 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042865409 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.844950+0000", 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8670000000000004e-05 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845013+0000", 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3659999999999997e-05 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.573 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103284 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 523264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.706082721000001, 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043179033999999998, 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103284 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802011+0000", 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802010+0000", 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802014+0000", 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1099999999999999e-06 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802021+0000", 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7729999999999999e-06 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.802027+0000", 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0279999999999997e-06 2024-09-13T05:00:12.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845053+0000", 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043025270999999997 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845130+0000", 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7387999999999998e-05 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.845190+0000", 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9938000000000001e-05 2024-09-13T05:00:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103531 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 387072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.698009872999997, 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.046484202000000002, 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103531 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810084+0000", 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810083+0000", 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810087+0000", 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5090000000000002e-06 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810114+0000", 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7263999999999998e-05 2024-09-13T05:00:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810122+0000", 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8310000000000001e-06 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851361+0000", 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.041238969 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.851601+0000", 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024058400000000001 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.856568+0000", 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049666449999999996 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103537 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.697686996000002, 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042298980999999999, 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103537 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810407+0000", 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810406+0000", 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810409+0000", 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0489999999999999e-06 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810417+0000", 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.763e-06 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810423+0000", 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3779999999999998e-06 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852568+0000", 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042144632000000001 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852644+0000", 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5908000000000006e-05 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852706+0000", 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1713999999999999e-05 2024-09-13T05:00:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103538 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 484352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.697646712999997, 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042420402000000003, 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103538 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810447+0000", 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810446+0000", 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810457+0000", 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0686999999999999e-05 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810486+0000", 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8784000000000001e-05 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810494+0000", 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5979999999999997e-06 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852727+0000", 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042232984000000001 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852801+0000", 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3691999999999998e-05 2024-09-13T05:00:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852867+0000", 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6391999999999997e-05 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103539 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.697591527999997, 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042532474000000001, 2024-09-13T05:00:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103539 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810502+0000", 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810501+0000", 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810506+0000", 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1740000000000002e-06 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810515+0000", 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9099999999999994e-06 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810521+0000", 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5359999999999998e-06 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852890+0000", 2024-09-13T05:00:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042368994 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.852968+0000", 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8097999999999997e-05 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853035+0000", 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6501999999999994e-05 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:103540 3.5 3:a757be47:::rbd_data.112a28ed7f25.0000000000000032:2a [read 510976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.697566623, 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042686832000000001, 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 103540 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810527+0000", 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810530+0000", 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.88e-06 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810538+0000", 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1729999999999995e-06 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.810544+0000", 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2179999999999996e-06 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853061+0000", 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.042516929000000002 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853135+0000", 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4333999999999997e-05 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:35.853214+0000", 2024-09-13T05:00:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8715999999999996e-05 2024-09-13T05:00:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:12.600 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:12.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:00:12.602 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:12.598+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:12.702 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:12.698+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:12.777 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:12.778 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:12.778 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T05:00:12.778 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:12.792 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:00:12.804 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:12.801+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:12.905 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:12.898+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:12.963 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:12.963 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:12.963 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T05:00:12.963 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T05:00:12.963 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:12.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:00:13.005 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:13.005+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:13.096 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T05:00:13.106 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:13.101+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:13.157 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:13.158 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:13.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:13.159 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:13.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.159 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T05:00:13.159 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:13.159 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.99119281599999, 2024-09-13T05:00:13.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T05:00:13.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T05:00:13.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T05:00:13.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T05:00:13.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T05:00:13.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.161 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:13.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:13.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T05:00:13.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:13.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T05:00:13.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T05:00:13.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T05:00:13.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T05:00:13.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T05:00:13.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T05:00:13.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T05:00:13.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T05:00:13.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T05:00:13.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T05:00:13.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T05:00:13.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T05:00:13.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T05:00:13.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T05:00:13.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T05:00:13.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.171 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.171 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T05:00:13.172 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:13.172 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.98875215699999, 2024-09-13T05:00:13.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T05:00:13.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:13.173 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:13.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:13.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T05:00:13.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:13.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T05:00:13.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T05:00:13.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T05:00:13.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T05:00:13.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T05:00:13.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T05:00:13.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T05:00:13.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T05:00:13.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T05:00:13.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T05:00:13.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:13.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T05:00:13.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T05:00:13.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T05:00:13.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T05:00:13.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.183 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:13.183 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:13.183 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 103.141920216, 2024-09-13T05:00:13.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T05:00:13.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:13.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T05:00:13.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.184 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T05:00:13.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:13.190 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 103.127409164, 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T05:00:13.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T05:00:13.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T05:00:13.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T05:00:13.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T05:00:13.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:00:13.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T05:00:13.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T05:00:13.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T05:00:13.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T05:00:13.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T05:00:13.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T05:00:13.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T05:00:13.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:13.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T05:00:13.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T05:00:13.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:13.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T05:00:13.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T05:00:13.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:13.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T05:00:13.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T05:00:13.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T05:00:13.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T05:00:13.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.201 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.202 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99140 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:13.202 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:13.202 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 40.785685461, 2024-09-13T05:00:13.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058811783999999999, 2024-09-13T05:00:13.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:13.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.203 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99140 2024-09-13T05:00:13.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.204 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:13.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:13.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369483+0000", 2024-09-13T05:00:13.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T05:00:13.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371901+0000", 2024-09-13T05:00:13.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024177479999999999 2024-09-13T05:00:13.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371935+0000", 2024-09-13T05:00:13.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999999e-05 2024-09-13T05:00:13.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371949+0000", 2024-09-13T05:00:13.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3917000000000001e-05 2024-09-13T05:00:13.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372078+0000", 2024-09-13T05:00:13.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012926999999999999 2024-09-13T05:00:13.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372339+0000", 2024-09-13T05:00:13.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000260394 2024-09-13T05:00:13.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:13.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.373383+0000", 2024-09-13T05:00:13.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010445809999999999 2024-09-13T05:00:13.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:13.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428034+0000", 2024-09-13T05:00:13.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054651138000000002 2024-09-13T05:00:13.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:13.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428093+0000", 2024-09-13T05:00:13.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8604000000000003e-05 2024-09-13T05:00:13.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:13.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428253+0000", 2024-09-13T05:00:13.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016013 2024-09-13T05:00:13.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428297+0000", 2024-09-13T05:00:13.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4459000000000002e-05 2024-09-13T05:00:13.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.217 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.217 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99141 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:13.218 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:13.218 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 40.783202029000002, 2024-09-13T05:00:13.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058946757000000002, 2024-09-13T05:00:13.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:13.219 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.219 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.219 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99141 2024-09-13T05:00:13.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.219 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:13.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:13.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371967+0000", 2024-09-13T05:00:13.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:13.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371974+0000", 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2609999999999999e-06 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371981+0000", 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371986+0000", 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3630000000000002e-06 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424374+0000", 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.052387356000000003 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424479+0000", 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010508199999999999 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424690+0000", 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021145300000000001 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429820+0000", 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051295810000000002 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430829+0000", 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010095480000000001 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430878+0000", 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8979e-05 2024-09-13T05:00:13.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430916+0000", 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7413000000000002e-05 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106233 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1172480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033393466, 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054064492999999998, 2024-09-13T05:00:13.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106233 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121777+0000", 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121793+0000", 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5945999999999999e-05 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121798+0000", 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3030000000000001e-06 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121801+0000", 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.898e-06 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175652+0000", 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053851796 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175743+0000", 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1002999999999996e-05 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175842+0000", 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8745000000000004e-05 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.230 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106244 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1191936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033367595999998, 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054259089000000003, 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106244 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.231 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121803+0000", 2024-09-13T05:00:13.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121805+0000", 2024-09-13T05:00:13.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.53e-06 2024-09-13T05:00:13.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121808+0000", 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.96e-06 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121810+0000", 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175871+0000", 2024-09-13T05:00:13.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054060460999999997 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175956+0000", 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5227000000000002e-05 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176063+0000", 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106764 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106245 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1205248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033358816000003, 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054466459000000002, 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106245 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:13.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:13.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:13.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121814+0000", 2024-09-13T05:00:13.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4079999999999999e-06 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121816+0000", 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8880000000000001e-06 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121819+0000", 2024-09-13T05:00:13.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176099+0000", 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054279922000000001 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176176+0000", 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7495000000000003e-05 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176279+0000", 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102503 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106246 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1218560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033350128999999, 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054660164999999997, 2024-09-13T05:00:13.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106246 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121822+0000", 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.342e-06 2024-09-13T05:00:13.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121825+0000", 2024-09-13T05:00:13.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.005e-06 2024-09-13T05:00:13.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121828+0000", 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.807e-06 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176305+0000", 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054476681999999998 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176387+0000", 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2238999999999996e-05 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176481+0000", 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4287e-05 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.298 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106247 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1231872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.298 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:13.298 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033341018999998, 2024-09-13T05:00:13.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054861114000000002, 2024-09-13T05:00:13.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106247 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121831+0000", 2024-09-13T05:00:13.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3209999999999999e-06 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121834+0000", 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121837+0000", 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176507+0000", 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054670165 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176591+0000", 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3820999999999996e-05 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176691+0000", 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100739 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106258 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1384448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033332315999999, 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055059518000000002, 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106258 2024-09-13T05:00:13.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121840+0000", 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2890000000000001e-06 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121843+0000", 2024-09-13T05:00:13.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8700000000000001e-06 2024-09-13T05:00:13.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121845+0000", 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6869999999999999e-06 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176717+0000", 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054871862 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176801+0000", 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3473999999999994e-05 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176898+0000", 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7535999999999996e-05 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106259 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1397760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.309 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033322251999998, 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055256324000000002, 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106259 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:13.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121851+0000", 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9920000000000002e-06 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121856+0000", 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5090000000000001e-06 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121861+0000", 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176928+0000", 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055067656999999999 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177010+0000", 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1298000000000007e-05 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177105+0000", 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5623000000000002e-05 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106260 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033307784999998, 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055448655999999999, 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106260 2024-09-13T05:00:13.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121865+0000", 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.553e-06 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121868+0000", 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.106e-06 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121870+0000", 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.481e-06 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177133+0000", 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055262349000000002 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177212+0000", 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9305e-05 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177312+0000", 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100083 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106261 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1424384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:13.319 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033298741000003, 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055627715000000001, 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106261 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:13.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121874+0000", 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3379999999999999e-06 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121876+0000", 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.937e-06 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121879+0000", 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4509999999999999e-06 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177331+0000", 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055451584999999998 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177401+0000", 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0269000000000001e-05 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177500+0000", 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9335999999999999e-05 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106262 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1437696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.033289992, 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055839664999999997, 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.324 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106262 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121883+0000", 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.823e-06 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121888+0000", 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121893+0000", 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.425e-06 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177529+0000", 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05563651 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177616+0000", 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6794999999999998e-05 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177721+0000", 2024-09-13T05:00:13.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104991 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106284 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1245184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.031863536000003, 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054629361000000001, 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106284 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123307+0000", 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123312+0000", 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9640000000000002e-06 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123324+0000", 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1965e-05 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123332+0000", 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0979999999999994e-06 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177750+0000", 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054418375999999997 2024-09-13T05:00:13.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177832+0000", 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1947999999999995e-05 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177937+0000", 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010498299999999999 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106285 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1258496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.031831693000001, 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054821169000000003, 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106285 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123342+0000", 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3519999999999998e-06 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123350+0000", 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6159999999999996e-06 2024-09-13T05:00:13.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123355+0000", 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2050000000000001e-06 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177969+0000", 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054613802000000003 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178053+0000", 2024-09-13T05:00:13.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3800000000000004e-05 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178161+0000", 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107876 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106286 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:13.338 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.024936242000003, 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054693622999999997, 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106286 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:13.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130234+0000", 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130240+0000", 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1519999999999997e-06 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130255+0000", 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4739000000000001e-05 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130262+0000", 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5979999999999999e-06 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178190+0000", 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047928040999999998 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178285+0000", 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5082999999999997e-05 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.184929+0000", 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066430589999999998 2024-09-13T05:00:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107657 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1654784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 36.943516703999997, 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054578494999999998, 2024-09-13T05:00:13.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107657 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211657+0000", 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1920000000000001e-06 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211674+0000", 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6923e-05 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211691+0000", 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6957e-05 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:13.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262566+0000", 2024-09-13T05:00:13.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050875110000000001 2024-09-13T05:00:13.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:13.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262650+0000", 2024-09-13T05:00:13.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.266233+0000", 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035824989999999998 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:13.348 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:13.350 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:00:13.350 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:13.345+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:13.452 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:13.445+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:13.487 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:123039 3.1b 3:deef8a74:::rbd_header.112a2c87bb71:head [call rbd.get_data_pool in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e130)", 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T05:00:13.487233+0000", 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 0.000291346, 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00033701899999999998, 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.488 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 123039 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:13.487233+0000", 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:13.487233+0000", 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.489 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:13.487231+0000", 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:13.487242+0000", 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1454000000000001e-05 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:13.487272+0000", 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9583000000000001e-05 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:13.487288+0000", 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6121999999999999e-05 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:13.487346+0000", 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7688000000000001e-05 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T05:00:13.487453+0000", 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106974 2024-09-13T05:00:13.491 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.492 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.492 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.492 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-13T05:00:13.492 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 1 2024-09-13T05:00:13.492 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:13.503 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:00:13.552 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:13.546+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:13.653 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:13.646+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:13.670 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:13.670 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:13.670 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:13.670 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:13.670 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:13.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:00:13.754 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:13.750+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:13.812 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.812 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.951003256999996, 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T05:00:13.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T05:00:13.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:13.817 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.950976346000004, 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.818 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T05:00:13.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.947230709999999, 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T05:00:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.943687385000004, 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T05:00:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.943647756999994, 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T05:00:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.943624096999997, 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.943536547999997, 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T05:00:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T05:00:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.943483892000003, 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T05:00:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.939558000999995, 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.853 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.854 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.857 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.939532376000003, 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T05:00:13.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.862 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.939498982000003, 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:13.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T05:00:13.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.934709053000006, 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T05:00:13.867 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.868 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T05:00:13.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.870 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T05:00:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T05:00:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T05:00:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T05:00:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.871 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T05:00:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T05:00:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.872 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.931314423999993, 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.873 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.874 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.875 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.876 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:13.877 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.931241748000005, 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:13.878 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T05:00:13.879 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.880 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.881 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.927499386999997, 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.882 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.883 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.884 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T05:00:13.885 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.886 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.927326395999998, 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.887 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T05:00:13.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T05:00:13.889 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.890 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.923483797000003, 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.891 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.892 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.893 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T05:00:13.894 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.895 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.883635819000006, 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.896 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.897 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.898 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.899 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.900 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.879306596999996, 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.901 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.902 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T05:00:13.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T05:00:13.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.903 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.903 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.903 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T05:00:13.903 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T05:00:13.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T05:00:13.904 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 79.871577596999998, 2024-09-13T05:00:13.905 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T05:00:13.907 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.908 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T05:00:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.910 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.910 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.910 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:13.910 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:13.910 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:13.911 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:00:13.912 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:13.905+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:14.011 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:14.005+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:14.089 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:14.089 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:14.089 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:14.103 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:00:14.145 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:14.141+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:14.213 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:14.209+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:14.270 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:14.270 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:14.270 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:14.270 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:14.270 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:14.284 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:00:14.315 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:14.310+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:14.416 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:14.410+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 118.41463423099999, 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.452 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.454 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T05:00:14.456 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 118.41458859399999, 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.457 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:14.458 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.459 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T05:00:14.460 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T05:00:14.461 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 118.41454050999999, 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.462 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.463 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.465 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T05:00:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 118.414444428, 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.467 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.468 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:14.469 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T05:00:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T05:00:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.472 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.473 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 118.414393057, 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T05:00:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T05:00:14.477 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T05:00:14.478 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 118.414347003, 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.479 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:14.480 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.481 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.482 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T05:00:14.483 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 118.41429694199999, 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.484 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.485 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.486 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T05:00:14.487 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T05:00:14.488 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 118.296881752, 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.489 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.490 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.491 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.492 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:14.493 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.494 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.205429024, 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.495 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.496 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T05:00:14.497 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T05:00:14.498 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.205347671, 2024-09-13T05:00:14.499 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T05:00:14.501 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.502 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.503 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.20527814099999, 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.504 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.505 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T05:00:14.506 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T05:00:14.507 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:14.508 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.205250865, 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:14.509 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.510 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T05:00:14.511 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T05:00:14.512 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.20515242, 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.513 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:14.514 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.515 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.516 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.517 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.20508285299999, 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:14.518 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.519 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.520 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.521 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.205012379, 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.522 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.523 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.524 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T05:00:14.525 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.526 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.204986694, 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.527 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.528 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.529 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.530 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.204951506, 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T05:00:14.531 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.532 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.533 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.535 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.20492699, 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.536 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.537 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T05:00:14.538 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.539 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.540 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 113.204857653, 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T05:00:14.541 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.542 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T05:00:14.543 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T05:00:14.544 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:99156 3.0 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:14.545 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 42.052071396999999, 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083885007999999997, 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 99156 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396500+0000", 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999933 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404915+0000", 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084155649999999999 2024-09-13T05:00:14.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404964+0000", 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8795000000000003e-05 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404987+0000", 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2719999999999999e-05 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405217+0000", 2024-09-13T05:00:14.548 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230163 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405435+0000", 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000218054 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 7", 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.406287+0000", 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00085163700000000003 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.470407+0000", 2024-09-13T05:00:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.064120636999999994 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480294+0000", 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098868949999999997 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480345+0000", 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1116999999999999e-05 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:14.550 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480391+0000", 2024-09-13T05:00:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6224999999999999e-05 2024-09-13T05:00:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:14.551 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:14.551 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:14.551 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:00:14.553 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:00:14.553 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:14.546+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:14.654 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:14.649+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:14.679 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:14.680 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:14.693 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:00:14.709 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:14.709 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:14.725 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:14.726 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:14.742 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:14.742 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:14.758 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:14.775 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:14.781 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:14.777+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:14.794 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:14.813 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:14.834 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:14.834 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:14.855 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:14.850+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:14.860 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:14.860 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:14.874 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:00:14.961 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:14.957+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:15.033 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:15.033 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:15.048 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:00:15.089 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:15.085+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:15.119 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:00:15.164 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:00:15.164 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:00:15.164 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-13T05:00:15.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-13T05:00:15.169 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:15.166+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:15.222 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:15.222 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:15.238 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:00:15.259 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:15.254+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:15.357 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:15.357 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:15.359 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:15.353+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:15.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:00:15.460 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:15.454+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:15.500 INFO:tasks.ceph.mon.a.smithi001.stderr:2024-09-13T05:00:15.494+0000 7f3f1af99640 -1 mon.a@0(leader).osd e132 definitely_dead 0 2024-09-13T05:00:15.533 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:15.533 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:15.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:00:15.560 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:15.557+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:15.661 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:15.661+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:15.678 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:15.678 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:15.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:00:15.763 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:15.758+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:15.819 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:15.819 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:15.833 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:00:15.864 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:15.861+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:15.964 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:15.958+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:16.002 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:16.002 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:16.016 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:00:16.065 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:16.061+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2024-09-13T05:00:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:marked in osd.1. 2024-09-13T05:00:16.150 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-13T05:00:16.166 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:16.161+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:16.200 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:16.200 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:16.215 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:00:16.267 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:16.261+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:16.334 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:16.334 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:16.349 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:00:16.397 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:16.394+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:16.469 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:16.465+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:16.493 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:00:16.581 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:16.578+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:16.665 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:16.666 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:16.671 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:16.666+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:16.680 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:00:16.773 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:16.769+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:16.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:00:16.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:00:16.899 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:16.894+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:16.974 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:16.970+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:17.033 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:17.033 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:17.040 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:17.040 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:17.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:00:17.057 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:17.057 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:17.074 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:17.074 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:17.074 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:17.073+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:17.090 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:17.090 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:17.115 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:17.116 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:17.136 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:17.136 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:17.157 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:17.157 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:17.175 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:17.170+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:17.177 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:17.177 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:17.181 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:17.181 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:17.181 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2024-09-13T05:00:17.181 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:17.198 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:00:17.277 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:17.273+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:17.367 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:17.367 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:17.367 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T05:00:17.367 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T05:00:17.368 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:17.377 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:17.373+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:17.382 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:00:17.478 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:17.474+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:17.504 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:17.504 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:17.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:17.504 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 143.33754313200001, 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T05:00:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T05:00:17.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T05:00:17.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.33510247300001, 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.488270532, 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T05:00:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.47375948, 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T05:00:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T05:00:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T05:00:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99140 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.132035776999999, 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058811783999999999, 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99140 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369483+0000", 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371901+0000", 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024177479999999999 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371935+0000", 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999999e-05 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371949+0000", 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3917000000000001e-05 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372078+0000", 2024-09-13T05:00:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012926999999999999 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372339+0000", 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000260394 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.373383+0000", 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010445809999999999 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428034+0000", 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054651138000000002 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428093+0000", 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8604000000000003e-05 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428253+0000", 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016013 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428297+0000", 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4459000000000002e-05 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99141 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.129552345, 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058946757000000002, 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99141 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371967+0000", 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371974+0000", 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2609999999999999e-06 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371981+0000", 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371986+0000", 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3630000000000002e-06 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424374+0000", 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.052387356000000003 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424479+0000", 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010508199999999999 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424690+0000", 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021145300000000001 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429820+0000", 2024-09-13T05:00:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051295810000000002 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430829+0000", 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010095480000000001 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430878+0000", 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8979e-05 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430916+0000", 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7413000000000002e-05 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106233 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1172480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379743781999998, 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054064492999999998, 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106233 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121777+0000", 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121793+0000", 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5945999999999999e-05 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121798+0000", 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3030000000000001e-06 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121801+0000", 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.898e-06 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175652+0000", 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053851796 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175743+0000", 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1002999999999996e-05 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175842+0000", 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8745000000000004e-05 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106244 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1191936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379717911999997, 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054259089000000003, 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106244 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121803+0000", 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121805+0000", 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.53e-06 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121808+0000", 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.96e-06 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121810+0000", 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-13T05:00:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175871+0000", 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054060460999999997 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175956+0000", 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5227000000000002e-05 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176063+0000", 2024-09-13T05:00:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106764 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106245 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1205248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379709132000002, 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054466459000000002, 2024-09-13T05:00:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106245 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:17.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121814+0000", 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4079999999999999e-06 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121816+0000", 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8880000000000001e-06 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121819+0000", 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176099+0000", 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054279922000000001 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176176+0000", 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7495000000000003e-05 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176279+0000", 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102503 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106246 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1218560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379700444999997, 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054660164999999997, 2024-09-13T05:00:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106246 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121822+0000", 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.342e-06 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121825+0000", 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.005e-06 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121828+0000", 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.807e-06 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176305+0000", 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054476681999999998 2024-09-13T05:00:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176387+0000", 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2238999999999996e-05 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176481+0000", 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4287e-05 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106247 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1231872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379691334999997, 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054861114000000002, 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106247 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121831+0000", 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3209999999999999e-06 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121834+0000", 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121837+0000", 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176507+0000", 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054670165 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176591+0000", 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3820999999999996e-05 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176691+0000", 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100739 2024-09-13T05:00:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106258 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1384448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379682631999998, 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055059518000000002, 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106258 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121840+0000", 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2890000000000001e-06 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121843+0000", 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8700000000000001e-06 2024-09-13T05:00:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121845+0000", 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6869999999999999e-06 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176717+0000", 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054871862 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176801+0000", 2024-09-13T05:00:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3473999999999994e-05 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176898+0000", 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7535999999999996e-05 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106259 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1397760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379672567999997, 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055256324000000002, 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106259 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121851+0000", 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9920000000000002e-06 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121856+0000", 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5090000000000001e-06 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121861+0000", 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176928+0000", 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055067656999999999 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177010+0000", 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1298000000000007e-05 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177105+0000", 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5623000000000002e-05 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106260 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379658100999997, 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055448655999999999, 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106260 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121865+0000", 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.553e-06 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121868+0000", 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.106e-06 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121870+0000", 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.481e-06 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177133+0000", 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055262349000000002 2024-09-13T05:00:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177212+0000", 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9305e-05 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177312+0000", 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100083 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106261 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1424384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379649057000002, 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055627715000000001, 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106261 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121874+0000", 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3379999999999999e-06 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121876+0000", 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.937e-06 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121879+0000", 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4509999999999999e-06 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177331+0000", 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055451584999999998 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177401+0000", 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0269000000000001e-05 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177500+0000", 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9335999999999999e-05 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106262 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1437696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.379640307999999, 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055839664999999997, 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106262 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121883+0000", 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.823e-06 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121888+0000", 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121893+0000", 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.425e-06 2024-09-13T05:00:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177529+0000", 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05563651 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177616+0000", 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6794999999999998e-05 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177721+0000", 2024-09-13T05:00:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104991 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106284 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1245184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.378213852000002, 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054629361000000001, 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106284 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123307+0000", 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123312+0000", 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9640000000000002e-06 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123324+0000", 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1965e-05 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123332+0000", 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0979999999999994e-06 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177750+0000", 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054418375999999997 2024-09-13T05:00:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177832+0000", 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1947999999999995e-05 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177937+0000", 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010498299999999999 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106285 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1258496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.378182009, 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054821169000000003, 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106285 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123342+0000", 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3519999999999998e-06 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123350+0000", 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6159999999999996e-06 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123355+0000", 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2050000000000001e-06 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177969+0000", 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054613802000000003 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178053+0000", 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3800000000000004e-05 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178161+0000", 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107876 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106286 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.371286558000001, 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054693622999999997, 2024-09-13T05:00:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106286 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130234+0000", 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130240+0000", 2024-09-13T05:00:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1519999999999997e-06 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130255+0000", 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4739000000000001e-05 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130262+0000", 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5979999999999999e-06 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178190+0000", 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047928040999999998 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178285+0000", 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5082999999999997e-05 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.184929+0000", 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066430589999999998 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107657 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1654784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 41.289867020000003, 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054578494999999998, 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107657 2024-09-13T05:00:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211657+0000", 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1920000000000001e-06 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211674+0000", 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6923e-05 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211691+0000", 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6957e-05 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262566+0000", 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050875110000000001 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262650+0000", 2024-09-13T05:00:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.266233+0000", 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035824989999999998 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:17.604 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:17.606 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:00:17.606 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:17.602+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:17.707 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:17.706+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:17.735 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:17.735 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:17.735 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:17.736 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:17.749 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:00:17.807 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:17.801+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:17.873 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:17.874 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:17.874 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:17.874 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:17.874 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:17.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:00:17.908 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:17.902+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:18.009 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:18.005+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:18.060 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.199408766000005, 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.061 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:18.062 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.063 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T05:00:18.064 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.065 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:18.066 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.199381854999999, 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:18.067 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.069 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T05:00:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.195636218999994, 2024-09-13T05:00:18.071 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:18.072 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.073 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T05:00:18.074 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T05:00:18.075 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.192092893999998, 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.076 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.078 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.079 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.080 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.192053266000002, 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.081 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:18.082 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.083 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.084 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.192029606000006, 2024-09-13T05:00:18.085 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.086 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T05:00:18.087 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.088 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.089 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.191942057000006, 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.090 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.091 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.092 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.093 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.191889400999997, 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T05:00:18.094 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.095 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T05:00:18.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.097 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.098 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.187963510000003, 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T05:00:18.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.100 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.101 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T05:00:18.102 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.187937884999997, 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T05:00:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.104 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T05:00:18.105 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.106 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T05:00:18.107 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.187904490999998, 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.108 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:18.109 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.110 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T05:00:18.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.112 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.183114562, 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.113 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T05:00:18.115 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.116 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.179719933000001, 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.117 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.118 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.119 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T05:00:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T05:00:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T05:00:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T05:00:18.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T05:00:18.121 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.179647256999999, 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.123 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.125 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T05:00:18.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.175904896000006, 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T05:00:18.127 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.128 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T05:00:18.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.130 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.131 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.175731905000006, 2024-09-13T05:00:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T05:00:18.132 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:18.133 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.134 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T05:00:18.135 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.136 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.171889305999997, 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T05:00:18.137 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.138 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T05:00:18.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.141 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.132041328, 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.142 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.143 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T05:00:18.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T05:00:18.145 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.146 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.127712106000004, 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T05:00:18.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T05:00:18.148 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.149 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.150 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.151 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 84.119983106000007, 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T05:00:18.152 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.153 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T05:00:18.154 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T05:00:18.155 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.156 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:18.157 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:00:18.158 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:00:18.159 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:18.153+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:18.259 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:18.253+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:18.344 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:18.344 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:18.344 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:18.344 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:18.360 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:00:18.360 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:18.353+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:18.461 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:18.457+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:18.527 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:18.527 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:18.527 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:18.528 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:18.528 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:18.542 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:00:18.585 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-09-13T05:00:18.581+0000 7ff2e47bf640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26358) UID: 0 2024-09-13T05:00:18.663 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:18.658+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:18.708 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.671566959, 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.709 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T05:00:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T05:00:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T05:00:18.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T05:00:18.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T05:00:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T05:00:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T05:00:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T05:00:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.715 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.671521322, 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T05:00:18.716 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:18.717 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T05:00:18.718 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T05:00:18.719 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.720 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.671473238, 2024-09-13T05:00:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T05:00:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.722 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.723 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.724 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.725 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.726 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T05:00:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T05:00:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.727 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.67137715600001, 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.728 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.729 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T05:00:18.730 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.731 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T05:00:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T05:00:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T05:00:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T05:00:18.732 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T05:00:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T05:00:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.733 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.67132578499999, 2024-09-13T05:00:18.734 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T05:00:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T05:00:18.735 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.736 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:18.737 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T05:00:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.738 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T05:00:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T05:00:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.739 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T05:00:18.740 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.741 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.671279731, 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.742 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T05:00:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.743 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.744 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.745 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.746 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.747 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T05:00:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T05:00:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.748 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.67122967, 2024-09-13T05:00:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T05:00:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.749 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.751 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.752 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T05:00:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T05:00:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.753 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.755 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 122.55381448, 2024-09-13T05:00:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T05:00:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.756 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T05:00:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.757 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.759 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T05:00:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T05:00:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.760 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T05:00:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T05:00:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.761 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T05:00:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T05:00:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T05:00:18.762 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T05:00:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T05:00:18.763 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T05:00:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T05:00:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T05:00:18.764 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T05:00:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T05:00:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T05:00:18.765 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T05:00:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T05:00:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.766 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.767 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.46236175200001, 2024-09-13T05:00:18.768 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:18.769 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.770 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T05:00:18.771 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T05:00:18.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.773 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.46228039899999, 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.774 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T05:00:18.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.779 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.462210869, 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.780 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.784 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T05:00:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T05:00:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T05:00:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T05:00:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.789 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.46218359300001, 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T05:00:18.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T05:00:18.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T05:00:18.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T05:00:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T05:00:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.462085148, 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.796 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:18.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T05:00:18.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T05:00:18.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T05:00:18.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.801 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.462015581, 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T05:00:18.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T05:00:18.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.46194510700001, 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T05:00:18.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T05:00:18.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T05:00:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.46191942199999, 2024-09-13T05:00:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T05:00:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:00:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T05:00:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T05:00:18.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.461884234, 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T05:00:18.818 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T05:00:18.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T05:00:18.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:00:18.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T05:00:18.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T05:00:18.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.461859718, 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.824 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 117.461790381, 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T05:00:18.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.832 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T05:00:18.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T05:00:18.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:99156 3.0 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 46.309004125000001, 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083885007999999997, 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 99156 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:18.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396500+0000", 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999933 2024-09-13T05:00:18.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404915+0000", 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084155649999999999 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404964+0000", 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8795000000000003e-05 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:18.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404987+0000", 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2719999999999999e-05 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405217+0000", 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230163 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405435+0000", 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000218054 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 7", 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.406287+0000", 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00085163700000000003 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.470407+0000", 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.064120636999999994 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480294+0000", 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098868949999999997 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480345+0000", 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1116999999999999e-05 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480391+0000", 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6224999999999999e-05 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:18.841 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:18.843 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:00:18.844 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:18.838+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:18.949 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2024-09-13T05:00:18.950 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:18.938+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:18.979 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:18.979 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:18.979 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:18.994 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:00:19.045 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:19.038+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:19.127 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:19.128 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:19.128 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:19.128 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:19.128 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:19.144 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:00:19.146 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:19.142+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:19.199 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:00:19.247 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:19.241+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:19.269 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:19.269 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:19.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:19.269 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:19.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.269 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.269 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468890414, 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.271 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T05:00:19.272 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.273 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46885616, 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.274 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.275 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.276 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T05:00:19.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T05:00:19.278 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468825259, 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.279 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T05:00:19.280 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.281 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T05:00:19.282 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.283 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46877337799999, 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T05:00:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.285 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T05:00:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.288 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468741575, 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.289 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T05:00:19.290 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.291 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.292 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T05:00:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468707622, 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.294 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.295 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T05:00:19.296 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T05:00:19.297 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468659886, 2024-09-13T05:00:19.298 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T05:00:19.299 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.300 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T05:00:19.301 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.302 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468620324, 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.303 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.304 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.305 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T05:00:19.306 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.307 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46857263, 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.308 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.309 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T05:00:19.310 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.311 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T05:00:19.312 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468509547, 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.313 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.314 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.315 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T05:00:19.316 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.317 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468473113, 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.318 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.319 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T05:00:19.320 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.321 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T05:00:19.322 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46843919, 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.323 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.324 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.325 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.326 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T05:00:19.327 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46837097700001, 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.328 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.329 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.330 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.331 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T05:00:19.332 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.468337271, 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.333 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.334 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T05:00:19.335 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.336 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.337 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46828707900001, 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.338 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:19.339 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.340 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.341 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46798104600001, 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T05:00:19.342 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.343 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T05:00:19.344 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.345 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T05:00:19.346 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46793184400001, 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.347 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.348 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.349 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T05:00:19.350 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.351 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.46788373, 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.352 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T05:00:19.353 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.354 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.355 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.467832253, 2024-09-13T05:00:19.356 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T05:00:19.357 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.358 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T05:00:19.359 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.360 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 115.467797172, 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T05:00:19.361 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.362 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.363 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T05:00:19.364 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:19.365 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T05:00:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T05:00:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:19.366 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:19.366 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:19.367 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:19.368 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:19.368 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:19.369 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:19.369 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:00:19.369 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:19.362+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:19.384 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:19.403 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:19.404 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:19.425 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:19.425 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:19.445 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:19.445 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:19.468 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:19.468 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:19.469 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:19.465+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:19.489 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:19.489 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:19.546 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:19.547 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:19.561 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:00:19.613 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:19.605+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:19.670 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:19.665+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:19.735 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:19.735 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:19.749 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:00:19.793 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:19.789+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:19.873 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:19.866+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:19.916 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:19.917 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:19.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:00:19.981 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:19.978+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:20.074 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:20.069+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:20.090 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:20.090 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:20.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:00:20.194 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:20.190+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:20.276 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:20.270+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:20.288 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:20.288 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:20.304 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:00:20.389 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:20.386+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:20.471 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:20.472 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:20.479 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:20.474+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:20.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:00:20.580 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:20.574+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:20.655 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:20.655 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:20.669 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:00:20.714 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:20.710+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:20.782 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:20.781+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:20.844 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:20.844 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:20.858 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:00:20.902 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:20.898+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:20.983 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:20.977+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:20.986 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:00:20.999 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:20.999 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:21.015 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:00:21.106 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:21.102+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:21.151 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:00:21.151 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:00:21.151 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T05:00:21.151 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.077269 2024-09-13T05:00:21.152 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0.0772689654145029 2024-09-13T05:00:21.190 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:21.190 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:21.191 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:21.186+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:21.206 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:00:21.286 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:21.281+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:21.371 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:21.371 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:21.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:00:21.387 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:21.381+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:21.488 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:21.482+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:21.506 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:21.507 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:21.510 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:00:21.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:00:21.609 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:21.606+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:21.659 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:21.660 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:21.676 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:21.676 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:21.689 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:21.686+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:21.690 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:21.690 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:21.692 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:21.692 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:21.706 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:00:21.709 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:21.709 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:21.729 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:21.750 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:21.771 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:21.790 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:21.786+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:21.792 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:21.792 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:21.826 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2024-09-13T05:00:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2024-09-13T05:00:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2024-09-13T05:00:21.827 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:21.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:00:21.929 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:21.926+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:21.955 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.7 primary-affinity to 0.077269 (850632) 2024-09-13T05:00:21.956 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T05:00:21.992 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:21.985+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:22.026 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2024-09-13T05:00:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2024-09-13T05:00:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2024-09-13T05:00:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e21)", 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 147.861177744, 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21311375799999999, 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3202668327", 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 2 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163978+0000", 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163976+0000", 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.163992+0000", 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5786000000000001e-05 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164013+0000", 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.105e-05 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164025+0000", 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1248e-05 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164146+0000", 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121587 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.164167+0000", 2024-09-13T05:00:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0950000000000001e-05 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.376864+0000", 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.21269670900000001 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:57:54.377092+0000", 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000228311 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_repop(client.4394.0:17093 3.17 e35/24 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head v 35'38, mlcod=35'35)", 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 125.858737085, 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.068825096000000002, 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166419+0000", 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.166418+0000", 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176888+0000", 2024-09-13T05:00:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010470070999999999 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176911+0000", 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3147999999999999e-05 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176919+0000", 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5150000000000006e-06 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.176983+0000", 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3057000000000003e-05 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.177010+0000", 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7419999999999998e-05 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235182+0000", 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058172117000000002 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:16.235244+0000", 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2039999999999996e-05 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40706 3.a 3:5206db42:::rbd_data.112aa73d3d8d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9853594] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 112.011905144, 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05412061, 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40706 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013251+0000", 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.013249+0000", 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018719+0000", 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054696249999999997 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018749+0000", 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0215000000000001e-05 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018763+0000", 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4076e-05 2024-09-13T05:00:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018841+0000", 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7669000000000005e-05 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.018962+0000", 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120457 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 1", 2024-09-13T05:00:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.019699+0000", 2024-09-13T05:00:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000737329 2024-09-13T05:00:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.063518+0000", 2024-09-13T05:00:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043818892999999998 2024-09-13T05:00:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067289+0000", 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037712940000000001 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067328+0000", 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.067372+0000", 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3423000000000001e-05 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:40710 3.9 3:943ba945:::rbd_data.112aa73d3d8d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.99739409199999, 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.075674509000000001, 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 40710 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027762+0000", 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.027760+0000", 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030148+0000", 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023871869999999998 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030174+0000", 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6846e-05 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030188+0000", 2024-09-13T05:00:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030269+0000", 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1026999999999995e-05 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.030454+0000", 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018531 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 4", 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.031089+0000", 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063499300000000004 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.075330+0000", 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.044241005999999999 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103368+0000", 2024-09-13T05:00:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028038317 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103409+0000", 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0725000000000001e-05 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:58:30.103436+0000", 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99140 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.655670389000001, 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058811783999999999, 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99140 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369486+0000", 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.369483+0000", 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999971 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371901+0000", 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024177479999999999 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371935+0000", 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4249999999999999e-05 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371949+0000", 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3917000000000001e-05 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372078+0000", 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012926999999999999 2024-09-13T05:00:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.372339+0000", 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000260394 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.373383+0000", 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010445809999999999 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428034+0000", 2024-09-13T05:00:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054651138000000002 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428093+0000", 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8604000000000003e-05 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428253+0000", 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016013 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.428297+0000", 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4459000000000002e-05 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:99141 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:head [delete] snapc 29=[29] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.653186957000003, 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.058946757000000002, 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 99141 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371969+0000", 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371967+0000", 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999981 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371974+0000", 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2609999999999999e-06 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371981+0000", 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.371986+0000", 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3630000000000002e-06 2024-09-13T05:00:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424374+0000", 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.052387356000000003 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424479+0000", 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010508199999999999 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for subops from 0", 2024-09-13T05:00:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.424690+0000", 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021145300000000001 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op_commit", 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.429820+0000", 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051295810000000002 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430829+0000", 2024-09-13T05:00:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010095480000000001 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430878+0000", 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8979e-05 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:32.430916+0000", 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7413000000000002e-05 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106233 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1172480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903378394000001, 2024-09-13T05:00:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054064492999999998, 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106233 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121778+0000", 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121777+0000", 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121793+0000", 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5945999999999999e-05 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121798+0000", 2024-09-13T05:00:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3030000000000001e-06 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121801+0000", 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.898e-06 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175652+0000", 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053851796 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175743+0000", 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1002999999999996e-05 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175842+0000", 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8745000000000004e-05 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106244 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1191936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903352523999999, 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054259089000000003, 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106244 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121804+0000", 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121803+0000", 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121805+0000", 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.53e-06 2024-09-13T05:00:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121808+0000", 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.96e-06 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121810+0000", 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4380000000000002e-06 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175871+0000", 2024-09-13T05:00:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054060460999999997 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.175956+0000", 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5227000000000002e-05 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176063+0000", 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106764 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106245 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1205248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903343743999997, 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054466459000000002, 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106245 2024-09-13T05:00:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121812+0000", 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121814+0000", 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4079999999999999e-06 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121816+0000", 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8880000000000001e-06 2024-09-13T05:00:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121819+0000", 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4710000000000001e-06 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176099+0000", 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054279922000000001 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176176+0000", 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7495000000000003e-05 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176279+0000", 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102503 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106246 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1218560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903335057, 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054660164999999997, 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106246 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121821+0000", 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121822+0000", 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.342e-06 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121825+0000", 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.005e-06 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121828+0000", 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.807e-06 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176305+0000", 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054476681999999998 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176387+0000", 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2238999999999996e-05 2024-09-13T05:00:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176481+0000", 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4287e-05 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106247 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1231872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903325946999999, 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054861114000000002, 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106247 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121830+0000", 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121831+0000", 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3209999999999999e-06 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121834+0000", 2024-09-13T05:00:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.836e-06 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121837+0000", 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176507+0000", 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054670165 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176591+0000", 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3820999999999996e-05 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176691+0000", 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100739 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106258 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1384448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903317244, 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055059518000000002, 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106258 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121839+0000", 2024-09-13T05:00:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121840+0000", 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2890000000000001e-06 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121843+0000", 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8700000000000001e-06 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121845+0000", 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6869999999999999e-06 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176717+0000", 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054871862 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176801+0000", 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3473999999999994e-05 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176898+0000", 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7535999999999996e-05 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106259 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1397760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903307179999999, 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055256324000000002, 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106259 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121849+0000", 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121851+0000", 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9920000000000002e-06 2024-09-13T05:00:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121856+0000", 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5090000000000001e-06 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121861+0000", 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5029999999999999e-06 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.176928+0000", 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055067656999999999 2024-09-13T05:00:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177010+0000", 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1298000000000007e-05 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177105+0000", 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5623000000000002e-05 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106260 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1411072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903292712999999, 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055448655999999999, 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106260 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121863+0000", 2024-09-13T05:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121865+0000", 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.553e-06 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121868+0000", 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.106e-06 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121870+0000", 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.481e-06 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177133+0000", 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055262349000000002 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177212+0000", 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9305e-05 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177312+0000", 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100083 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106261 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1424384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903283668999997, 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055627715000000001, 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106261 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121872+0000", 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121874+0000", 2024-09-13T05:00:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3379999999999999e-06 2024-09-13T05:00:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121876+0000", 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.937e-06 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121879+0000", 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4509999999999999e-06 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177331+0000", 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055451584999999998 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177401+0000", 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0269000000000001e-05 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177500+0000", 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9335999999999999e-05 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106262 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1437696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.903274920000001, 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055839664999999997, 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106262 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121881+0000", 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967296 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121883+0000", 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.823e-06 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121888+0000", 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3249999999999998e-06 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.121893+0000", 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.425e-06 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177529+0000", 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.05563651 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177616+0000", 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6794999999999998e-05 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177721+0000", 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104991 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106284 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1245184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.901848463999997, 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054629361000000001, 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106284 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123308+0000", 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123307+0000", 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123312+0000", 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9640000000000002e-06 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123324+0000", 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1965e-05 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123332+0000", 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0979999999999994e-06 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177750+0000", 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054418375999999997 2024-09-13T05:00:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177832+0000", 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1947999999999995e-05 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177937+0000", 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010498299999999999 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106285 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1258496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.901816621000002, 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054821169000000003, 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106285 2024-09-13T05:00:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123339+0000", 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123342+0000", 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3519999999999998e-06 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123350+0000", 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6159999999999996e-06 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.123355+0000", 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2050000000000001e-06 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.177969+0000", 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054613802000000003 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178053+0000", 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3800000000000004e-05 2024-09-13T05:00:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178161+0000", 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107876 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:106286 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.894921170000003, 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054693622999999997, 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 106286 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130235+0000", 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130234+0000", 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.999999 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130240+0000", 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1519999999999997e-06 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130255+0000", 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4739000000000001e-05 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.130262+0000", 2024-09-13T05:00:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5979999999999999e-06 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178190+0000", 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.047928040999999998 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.178285+0000", 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5082999999999997e-05 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.184929+0000", 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066430589999999998 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4394.0:107657 3.1d 3:b99f9bbd:::rbd_data.112afaf35b85.0000000000000000:29 [read 1654784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.813501631999998, 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054578494999999998, 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2024-09-13T05:00:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4394", 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 107657 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211654+0000", 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211657+0000", 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1920000000000001e-06 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2024-09-13T05:00:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211674+0000", 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6923e-05 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.211691+0000", 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6957e-05 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262566+0000", 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.050875110000000001 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.262650+0000", 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4370999999999996e-05 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2024-09-13T04:59:36.266233+0000", 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035824989999999998 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-13T05:00:22.131 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:22.133 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:00:22.134 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:22.129+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:22.234 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:22.229+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:22.265 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:22.265 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:22.265 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:22.265 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:22.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:00:22.334 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-09-13T05:00:22.330+0000 7f92c2dfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26176) UID: 0 2024-09-13T05:00:22.435 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:22.429+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:22.443 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:22.443 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:22.443 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:22.443 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:22.443 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:22.457 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:00:22.537 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-09-13T05:00:22.530+0000 7faf82bee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26178) UID: 0 2024-09-13T05:00:22.624 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78796 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5083136~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.763567038000005, 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103076531, 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.625 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78796 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858001+0000", 2024-09-13T05:00:22.626 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858000+0000", 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858004+0000", 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4860000000000002e-06 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.627 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858015+0000", 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0896999999999999e-05 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858022+0000", 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5480000000000004e-06 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960755+0000", 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102733003 2024-09-13T05:00:22.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.960886+0000", 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013135299999999999 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961078+0000", 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019101599999999999 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.629 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78797 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5087232~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.763540126999999, 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10340223599999999, 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78797 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.630 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858028+0000", 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858027+0000", 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.631 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858030+0000", 2024-09-13T05:00:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.632 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858041+0000", 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0671e-05 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.858048+0000", 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6479999999999997e-06 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961150+0000", 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103102503 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.633 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961276+0000", 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00012540699999999999 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.961430+0000", 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154427 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78806 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5107712~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.634 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.759794490999994, 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103143454, 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78806 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:22.635 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861774+0000", 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861773+0000", 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.636 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861779+0000", 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0769999999999996e-06 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861812+0000", 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.3228000000000001e-05 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.861826+0000", 2024-09-13T05:00:22.637 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4236999999999999e-05 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964606+0000", 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102779551 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964749+0000", 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014360999999999999 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.638 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.964917+0000", 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016774300000000001 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78814 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5451776~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.756251165999998, 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103074798, 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.639 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78814 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.640 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865317+0000", 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865316+0000", 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865328+0000", 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.132e-05 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.641 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865337+0000", 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.7140000000000005e-06 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865349+0000", 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1551e-05 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968091+0000", 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102742377 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.642 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968223+0000", 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013168000000000001 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968392+0000", 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016871 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.643 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78815 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5455872~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.756211538000002, 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10345816100000001, 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.644 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78815 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.645 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865356+0000", 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865359+0000", 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.4709999999999999e-06 2024-09-13T05:00:22.646 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865367+0000", 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.8690000000000004e-06 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865374+0000", 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.0049999999999998e-06 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.647 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968485+0000", 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103110586 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968630+0000", 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014480999999999999 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968815+0000", 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018498000000000001 2024-09-13T05:00:22.648 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78816 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5459968~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.756187878000006, 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10383574199999999, 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.649 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78816 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.650 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865380+0000", 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865383+0000", 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.18e-06 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865404+0000", 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1759e-05 2024-09-13T05:00:22.651 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865421+0000", 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6331e-05 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.968890+0000", 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103469008 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969021+0000", 2024-09-13T05:00:22.652 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000130777 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969216+0000", 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019526999999999999 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78817 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5464064~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.756100329000006, 2024-09-13T05:00:22.653 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10413759, 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78817 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.654 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865468+0000", 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865458+0000", 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999905 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865472+0000", 2024-09-13T05:00:22.655 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4185000000000001e-05 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865491+0000", 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8589000000000001e-05 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865506+0000", 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4816e-05 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.656 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969306+0000", 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103799774 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969449+0000", 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014290700000000001 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969605+0000", 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000156716 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.657 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78818 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5468160~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.756047672999998, 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104473465, 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78818 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.658 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865520+0000", 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865519+0000", 2024-09-13T05:00:22.659 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865525+0000", 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4820000000000002e-06 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865541+0000", 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.5807999999999998e-05 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.660 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.865556+0000", 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.526e-05 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969670+0000", 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.104113834 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969821+0000", 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015081599999999999 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.661 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.969994+0000", 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000173136 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78825 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5652480~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.752121782000003, 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103177379, 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.662 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78825 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.663 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869446+0000", 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869449+0000", 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2349999999999999e-06 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.664 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869459+0000", 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0411000000000001e-05 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869466+0000", 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3849999999999999e-06 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972323+0000", 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102857828 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.665 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972460+0000", 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013673 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972624+0000", 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016339300000000001 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78826 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5660672~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.752096156999997, 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103528395, 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78826 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869472+0000", 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869471+0000", 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.668 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869483+0000", 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1565e-05 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869493+0000", 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.8490000000000001e-06 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869499+0000", 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4169999999999997e-06 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.669 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972697+0000", 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103198132 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.972832+0000", 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013484400000000001 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973000+0000", 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016813599999999999 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.670 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78827 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5656576~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.752062762999998, 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10398974599999999, 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78827 2024-09-13T05:00:22.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.672 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869505+0000", 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869508+0000", 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869516+0000", 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.0050000000000004e-06 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.673 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.869522+0000", 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1369999999999999e-06 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973126+0000", 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103604273 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973262+0000", 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013554300000000001 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.973495+0000", 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000233248 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78837 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5693440~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.747272834, 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103157085, 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.675 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78837 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.676 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874295+0000", 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.677 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874298+0000", 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.965e-06 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874308+0000", 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0203e-05 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.678 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.874314+0000", 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4509999999999997e-06 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977099+0000", 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102784608 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977256+0000", 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000157119 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.679 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.977452+0000", 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000196276 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78844 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6086656~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.743878205000001, 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103097944, 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.680 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78844 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.681 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877690+0000", 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877689+0000", 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877696+0000", 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3080000000000002e-06 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.682 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877728+0000", 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2147999999999998e-05 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877746+0000", 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8068e-05 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980457+0000", 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102710198 2024-09-13T05:00:22.683 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980601+0000", 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014431700000000001 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980788+0000", 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018679600000000001 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.684 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78845 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 6037504~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.743805528999999, 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10342493899999999, 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78845 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.685 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877762+0000", 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877761+0000", 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.686 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877767+0000", 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9859999999999999e-06 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877799+0000", 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.171e-05 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.687 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.877824+0000", 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4576999999999999e-05 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980863+0000", 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303936599999999 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.980995+0000", 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013188300000000001 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.688 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.981187+0000", 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019252599999999999 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78854 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5857280~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.740063168000006, 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10324399400000001, 2024-09-13T05:00:22.689 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78854 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.690 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881505+0000", 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881504+0000", 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881547+0000", 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.2911000000000003e-05 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.691 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881557+0000", 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9389999999999992e-06 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881564+0000", 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1770000000000001e-06 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984447+0000", 2024-09-13T05:00:22.692 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10288267099999999 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984579+0000", 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013206099999999999 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984749+0000", 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00016980900000000001 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.693 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78855 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5861376~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.739890177000007, 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10346275100000001, 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78855 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.694 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881678+0000", 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881571+0000", 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9998927 2024-09-13T05:00:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881711+0000", 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000140063 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881782+0000", 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1364000000000003e-05 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.881799+0000", 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6977e-05 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984823+0000", 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10302415099999999 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.984953+0000", 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129701 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.697 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.985141+0000", 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018773600000000001 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78864 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 5890048~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.736047577999997, 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.103154778, 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.698 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78864 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.699 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885520+0000", 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885529+0000", 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1370000000000008e-06 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885537+0000", 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.7880000000000007e-06 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.885543+0000", 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2489999999999998e-06 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988362+0000", 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10281879100000001 2024-09-13T05:00:22.701 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988496+0000", 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013438999999999999 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.988675+0000", 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000179006 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.702 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78963 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9568256~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.6961996, 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10303420000000001, 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78963 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.703 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925368+0000", 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925371+0000", 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.1109999999999999e-06 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925379+0000", 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.5859999999999999e-06 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.925385+0000", 2024-09-13T05:00:22.705 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2600000000000002e-06 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028155+0000", 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102770545 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028261+0000", 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000106129 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.706 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.028403+0000", 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014116000000000001 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78975 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 9981952~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.691870378000004, 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.102968041, 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.707 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78975 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929698+0000", 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.708 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929602+0000", 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999042 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929702+0000", 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100212 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929725+0000", 2024-09-13T05:00:22.709 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2904999999999998e-05 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.929757+0000", 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.2353e-05 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032413+0000", 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10265519300000001 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.710 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032521+0000", 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010830100000000001 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.032666+0000", 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014479400000000001 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:78996 3.19 3:9dd72a81:::rbd_data.112a82343b36.0000000000000000:11 [read 10399744~4096] snapc 0=[] RETRY=5 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:22.711 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 88.684141378000007, 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10322122, 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 78996 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:22.712 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937427+0000", 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937426+0000", 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:22.713 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937438+0000", 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2157000000000001e-05 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937448+0000", 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0151e-05 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:53.937455+0000", 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4570000000000002e-06 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:22.714 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040401+0000", 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10294579800000001 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040495+0000", 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4689000000000001e-05 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:54.040648+0000", 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015253099999999999 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.715 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:22.716 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:22.716 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:22.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:00:22.718 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:22.714+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:22.818 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:22.818+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:22.842 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:22.843 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:22.843 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:22.843 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:22.857 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:00:22.919 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:22.913+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:22.989 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:22.990 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:22.990 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:22.990 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:22.990 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:23.004 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:00:23.049 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:23.045+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:23.120 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:23.113+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:23.178 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15754 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 782336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:23.178 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 127.141020346, 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081827163999999994, 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15754 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.179 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033944+0000", 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033943+0000", 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033949+0000", 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3400000000000003e-06 2024-09-13T05:00:23.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033966+0000", 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6612e-05 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033979+0000", 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115286+0000", 2024-09-13T05:00:23.181 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081306460999999997 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115501+0000", 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000215551 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115771+0000", 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00026960100000000001 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.182 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15755 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 823296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 127.14097470900001, 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082271844999999996, 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15755 2024-09-13T05:00:23.183 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033989+0000", 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.184 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033988+0000", 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.033995+0000", 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4470000000000003e-06 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034012+0000", 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6748999999999998e-05 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.185 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034026+0000", 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4331999999999999e-05 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.115856+0000", 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081830228000000005 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116034+0000", 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000177861 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.186 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116261+0000", 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00022713799999999999 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15756 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 864256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 127.14092662500001, 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082665054000000002, 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.187 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15756 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034037+0000", 2024-09-13T05:00:23.188 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034036+0000", 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034042+0000", 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8930000000000002e-06 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.189 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034058+0000", 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6226e-05 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034072+0000", 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3356000000000001e-05 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116345+0000", 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082273356000000006 2024-09-13T05:00:23.190 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116489+0000", 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000143893 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116702+0000", 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00021333999999999999 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.191 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15757 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 905216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 127.14083054300001, 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083307708999999994, 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15757 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034133+0000", 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034132+0000", 2024-09-13T05:00:23.193 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034139+0000", 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8290000000000002e-06 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034159+0000", 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.9715999999999999e-05 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034173+0000", 2024-09-13T05:00:23.194 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4259e-05 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116779+0000", 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082606225000000005 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.116971+0000", 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00019183700000000001 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117441+0000", 2024-09-13T05:00:23.195 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00047023300000000002 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15758 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 127.14077917199999, 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083731251000000007, 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.196 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15758 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034185+0000", 2024-09-13T05:00:23.197 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034184+0000", 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034190+0000", 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.9429999999999997e-06 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.198 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034206+0000", 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6486000000000001e-05 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034219+0000", 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3176000000000001e-05 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117476+0000", 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083256635999999995 2024-09-13T05:00:23.199 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117664+0000", 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018856799999999999 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117916+0000", 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00025161900000000002 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.200 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15759 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 987136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 127.140733118, 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083995004999999998, 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15759 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.201 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034231+0000", 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034230+0000", 2024-09-13T05:00:23.202 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034236+0000", 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8189999999999997e-06 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034252+0000", 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6308999999999999e-05 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.203 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034267+0000", 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4547e-05 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.117984+0000", 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083717689999999997 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118093+0000", 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010883900000000001 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.204 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118226+0000", 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00013270099999999999 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:15760 3.18 3:19f37292:::rbd_data.112af6c12117.0000000000000008:6 [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 127.140683057, 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.084390586000000004, 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.205 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 15760 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.206 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034281+0000", 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034280+0000", 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034286+0000", 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6300000000000003e-06 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.207 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034304+0000", 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.8249000000000001e-05 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.034317+0000", 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2952999999999999e-05 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118275+0000", 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083958659000000005 2024-09-13T05:00:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118464+0000", 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00018837500000000001 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.118672+0000", 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000207684 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.209 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:17093 3.17 3:ee061e05:::rbd_data.112ab86ccaf9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 31973021~64334] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 127.023267867, 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.10886562299999999, 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 17093 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151696+0000", 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.151694+0000", 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999976 2024-09-13T05:00:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.162965+0000", 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.011271056999999999 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163015+0000", 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9830000000000002e-05 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163036+0000", 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1413e-05 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163145+0000", 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010916699999999999 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.163423+0000", 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000277822 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.213 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 3", 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.165088+0000", 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.001664963 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.259439+0000", 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.094351138000000001 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260258+0000", 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00081908600000000001 2024-09-13T05:00:23.214 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260407+0000", 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014868600000000001 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:16.260562+0000", 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000154618 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.215 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31467 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1753088~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.93181513899999, 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081626708000000006, 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31467 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.216 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243149+0000", 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243148+0000", 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.217 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243152+0000", 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.286e-06 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243162+0000", 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0029e-05 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243169+0000", 2024-09-13T05:00:23.218 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2439999999999996e-06 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324606+0000", 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081437486000000003 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324688+0000", 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.1248999999999994e-05 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.219 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324776+0000", 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8020000000000001e-05 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31468 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1785856~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.931733786, 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081866147, 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.220 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31468 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243230+0000", 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.221 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243229+0000", 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243234+0000", 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3499999999999999e-06 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243244+0000", 2024-09-13T05:00:23.222 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.9229999999999997e-06 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243251+0000", 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.6290000000000001e-06 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324838+0000", 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081587344000000006 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.223 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.324939+0000", 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100911 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325096+0000", 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015695999999999999 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.224 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31469 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1818624~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.931664256, 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082000770000000001, 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31469 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.225 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243300+0000", 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243299+0000", 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.226 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243303+0000", 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.856e-06 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243312+0000", 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4229999999999994e-06 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243319+0000", 2024-09-13T05:00:23.227 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2230000000000001e-06 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325143+0000", 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.081823876000000004 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325212+0000", 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9251000000000005e-05 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.228 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325301+0000", 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.8016999999999994e-05 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31470 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1851392~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.93163697999999, 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082276804999999995, 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.229 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31470 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.230 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243327+0000", 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243326+0000", 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243330+0000", 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.557e-06 2024-09-13T05:00:23.231 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243338+0000", 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3389999999999993e-06 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243346+0000", 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.4000000000000003e-06 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325363+0000", 2024-09-13T05:00:23.232 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082017128999999994 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325459+0000", 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5707999999999999e-05 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325604+0000", 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000145296 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.233 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31471 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1884160~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.931538535, 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082391304999999998, 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31471 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.234 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243425+0000", 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.235 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243428+0000", 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8530000000000002e-06 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243438+0000", 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4460000000000007e-06 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243445+0000", 2024-09-13T05:00:23.236 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3200000000000002e-06 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325648+0000", 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082203087999999994 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325711+0000", 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.2971000000000004e-05 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.237 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325817+0000", 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010556999999999999 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31472 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1916928~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.931468968, 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082623413000000007, 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.238 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31472 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243495+0000", 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.239 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243494+0000", 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243498+0000", 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.827e-06 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243507+0000", 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0259999999999996e-06 2024-09-13T05:00:23.240 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243514+0000", 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2899999999999997e-06 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325872+0000", 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082357927999999997 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.325973+0000", 2024-09-13T05:00:23.241 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000100448 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326118+0000", 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00014579099999999999 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31473 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1949696~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:23.242 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.93139849400001, 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082724923000000006, 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31473 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.243 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243565+0000", 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243568+0000", 2024-09-13T05:00:23.244 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.8689999999999997e-06 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243578+0000", 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.3130000000000004e-06 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243585+0000", 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.2400000000000001e-06 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.245 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326141+0000", 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082556322000000001 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326205+0000", 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.3699000000000001e-05 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326290+0000", 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5439999999999995e-05 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.246 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31474 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 1982464~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.931372809, 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083024788000000002, 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31474 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.247 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.248 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243591+0000", 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243600+0000", 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.0659999999999993e-06 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243609+0000", 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243616+0000", 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.3560000000000004e-06 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326354+0000", 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082737677999999995 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326463+0000", 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00010990200000000001 2024-09-13T05:00:23.250 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326616+0000", 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00015250300000000001 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31475 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2015232~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.931337621, 2024-09-13T05:00:23.251 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083150616999999996, 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31475 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:23.252 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243626+0000", 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243625+0000", 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.253 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243629+0000", 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.6459999999999999e-06 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243637+0000", 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3629999999999994e-06 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243644+0000", 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.254 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326634+0000", 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.082989645000000001 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326694+0000", 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.0371000000000002e-05 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326777+0000", 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2502000000000004e-05 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.255 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31476 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2048000~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.931313105, 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083436629999999998, 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31476 2024-09-13T05:00:23.256 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243651+0000", 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243650+0000", 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243654+0000", 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.467e-06 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243665+0000", 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1225e-05 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.258 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243672+0000", 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.1300000000000003e-06 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326838+0000", 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083166157000000004 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.326958+0000", 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00011945700000000001 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.259 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327088+0000", 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000129784 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:31477 3.1f 3:fbe8647c:::rbd_data.112ab86ccaf9.0000000000000004:8 [read 2080768~16384] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 121.931243768, 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083593212, 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.260 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 31477 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.261 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243720+0000", 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243719+0000", 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243723+0000", 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0570000000000003e-06 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.262 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243733+0000", 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.4120000000000007e-06 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.243740+0000", 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.8839999999999997e-06 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327110+0000", 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083370305000000006 2024-09-13T05:00:23.263 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327172+0000", 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.1744999999999999e-05 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:21.327313+0000", 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000141786 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.264 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:99156 3.0 3:035c3f82:::rbd_data.112afaf35b85.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 50.778457512000003, 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.083885007999999997, 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 99156 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.265 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396506+0000", 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.266 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.396500+0000", 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999933 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404915+0000", 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0084155649999999999 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404964+0000", 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8795000000000003e-05 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.267 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.404987+0000", 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2719999999999999e-05 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405217+0000", 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000230163 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.405435+0000", 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.000218054 2024-09-13T05:00:23.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "waiting for subops from 7", 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.406287+0000", 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.00085163700000000003 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "op_commit", 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.470407+0000", 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.064120636999999994 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:00:23.269 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480294+0000", 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.0098868949999999997 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "commit_sent", 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480345+0000", 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1116999999999999e-05 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.270 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.271 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:59:32.480391+0000", 2024-09-13T05:00:23.271 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6224999999999999e-05 2024-09-13T05:00:23.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.271 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.271 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.271 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.271 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:23.272 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:00:23.273 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:23.270+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:23.374 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:23.370+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:23.444 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:23.444 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:23.444 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:23.459 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:00:23.474 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-09-13T05:00:23.474+0000 7f5467055640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26182) UID: 0 2024-09-13T05:00:23.574 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-09-13T05:00:23.569+0000 7f5a10686640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26360) UID: 0 2024-09-13T05:00:23.581 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:23.581 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:23.581 INFO:teuthology.orchestra.run.smithi144.stdout: "complaint_time": 30, 2024-09-13T05:00:23.582 INFO:teuthology.orchestra.run.smithi144.stdout: "num_blocked_ops": 0 2024-09-13T05:00:23.582 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:23.597 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:00:23.675 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-09-13T05:00:23.674+0000 7ffa4bfa3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26180) UID: 0 2024-09-13T05:00:23.770 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:23.770 INFO:teuthology.orchestra.run.smithi144.stdout: "size": 20, 2024-09-13T05:00:23.770 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 600, 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [ 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34416 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798694+0000", 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.96850328399999, 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060311929, 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34416 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798694+0000", 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798693+0000", 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798697+0000", 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.9829999999999998e-06 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798709+0000", 2024-09-13T05:00:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1352000000000001e-05 2024-09-13T05:00:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798718+0000", 2024-09-13T05:00:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.1479999999999995e-06 2024-09-13T05:00:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858916+0000", 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060198185000000001 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.858965+0000", 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.9282e-05 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859006+0000", 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0623000000000001e-05 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34418 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798728+0000", 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.96846902999999, 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060416742000000002, 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34418 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798728+0000", 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798727+0000", 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798732+0000", 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1180000000000002e-06 2024-09-13T05:00:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798743+0000", 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1712999999999999e-05 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798751+0000", 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2300000000000008e-06 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859038+0000", 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060286134999999998 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859087+0000", 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8983000000000002e-05 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859145+0000", 2024-09-13T05:00:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.8375000000000003e-05 2024-09-13T05:00:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34419 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6389760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798759+0000", 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.96843812900001, 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060556182, 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34419 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798759+0000", 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798762+0000", 2024-09-13T05:00:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 3.782e-06 2024-09-13T05:00:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798791+0000", 2024-09-13T05:00:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9192000000000001e-05 2024-09-13T05:00:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798802+0000", 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.009e-05 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859181+0000", 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060379295999999999 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859266+0000", 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5521999999999999e-05 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859315+0000", 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8909999999999998e-05 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.789 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34421 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798811+0000", 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.968386248, 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060640309000000003, 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.790 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34421 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.791 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798811+0000", 2024-09-13T05:00:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.792 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798810+0000", 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798815+0000", 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.3310000000000003e-06 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.793 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798827+0000", 2024-09-13T05:00:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1590000000000001e-05 2024-09-13T05:00:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798835+0000", 2024-09-13T05:00:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3969999999999994e-06 2024-09-13T05:00:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.794 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859350+0000", 2024-09-13T05:00:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060514501999999998 2024-09-13T05:00:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859396+0000", 2024-09-13T05:00:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6748e-05 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859451+0000", 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.4848000000000001e-05 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34423 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6455296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798843+0000", 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.968354445, 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060787960000000002, 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34423 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798843+0000", 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798842+0000", 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798846+0000", 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.34e-06 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798857+0000", 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0645000000000001e-05 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798866+0000", 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9679999999999995e-06 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859481+0000", 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060614595 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859545+0000", 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.4379000000000001e-05 2024-09-13T05:00:23.800 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859631+0000", 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.5779000000000007e-05 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34425 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6488064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798877+0000", 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.968320492, 2024-09-13T05:00:23.801 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060896737999999999, 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34425 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798877+0000", 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798876+0000", 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798881+0000", 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7550000000000002e-06 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.803 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798893+0000", 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.217e-05 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798915+0000", 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.2019e-05 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859668+0000", 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060753251000000001 2024-09-13T05:00:23.804 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859725+0000", 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.6762000000000003e-05 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859773+0000", 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8752999999999999e-05 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.805 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34427 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6520832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798924+0000", 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.968272756, 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060968742999999999, 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34427 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798924+0000", 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798923+0000", 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.807 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798928+0000", 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0830000000000003e-06 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798944+0000", 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.59e-05 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798954+0000", 2024-09-13T05:00:23.808 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0200999999999999e-05 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859804+0000", 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060850082 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859853+0000", 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8097999999999999e-05 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859893+0000", 2024-09-13T05:00:23.809 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0680999999999999e-05 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34429 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.798964+0000", 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.96823319400001, 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061065267999999999, 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.810 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34429 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798964+0000", 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.811 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798963+0000", 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798980+0000", 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6770999999999999e-05 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.798994+0000", 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3543000000000001e-05 2024-09-13T05:00:23.812 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799003+0000", 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2960000000000004e-06 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859923+0000", 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.060920407000000003 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.859979+0000", 2024-09-13T05:00:23.813 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5729999999999997e-05 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860029+0000", 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0176999999999997e-05 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34431 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6586368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799012+0000", 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.9681855, 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061172786999999999, 2024-09-13T05:00:23.814 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34431 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.815 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799012+0000", 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799011+0000", 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799041+0000", 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.9708000000000001e-05 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.816 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799054+0000", 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.3354e-05 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799066+0000", 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1833e-05 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860068+0000", 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061002580000000001 2024-09-13T05:00:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860134+0000", 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.5629000000000005e-05 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860185+0000", 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0426999999999997e-05 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34433 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799075+0000", 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.968122417, 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061246161, 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34433 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799075+0000", 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799074+0000", 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.820 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799079+0000", 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.1240000000000004e-06 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799090+0000", 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0987e-05 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799101+0000", 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1002000000000001e-05 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.821 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860217+0000", 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061116009999999998 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860274+0000", 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.7262000000000002e-05 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860321+0000", 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6782e-05 2024-09-13T05:00:23.822 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34435 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6651904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799111+0000", 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.96808598299999, 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061328369000000001, 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.823 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34435 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T05:00:23.824 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799111+0000", 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799115+0000", 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5940000000000004e-06 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.825 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799128+0000", 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.287e-05 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799138+0000", 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.5550000000000008e-06 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860351+0000", 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061213286999999998 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.826 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860398+0000", 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7086000000000003e-05 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860440+0000", 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1656999999999998e-05 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34437 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.827 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799145+0000", 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.96805206000001, 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061431973000000001, 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34437 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.828 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799145+0000", 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.829 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799149+0000", 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6959999999999996e-06 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799160+0000", 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.0905e-05 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799168+0000", 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.2220000000000002e-06 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.830 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860471+0000", 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061302307 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860526+0000", 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.5733000000000003e-05 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860577+0000", 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0769999999999997e-05 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.831 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34439 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6717440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799213+0000", 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.967983847, 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061481348999999998, 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34439 2024-09-13T05:00:23.832 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799213+0000", 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.833 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799212+0000", 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799218+0000", 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2939999999999996e-06 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799229+0000", 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1484999999999999e-05 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.834 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799238+0000", 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.4759999999999998e-06 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860607+0000", 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061369795999999997 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860653+0000", 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5670999999999997e-05 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.835 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860695+0000", 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1619000000000003e-05 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34441 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.836 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799247+0000", 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.967950141, 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061570973000000001, 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34441 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.837 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799247+0000", 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799246+0000", 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.838 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799250+0000", 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.1420000000000003e-06 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799277+0000", 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6582999999999999e-05 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799289+0000", 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1800000000000001e-05 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.839 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860725+0000", 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061435734999999998 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860771+0000", 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6628999999999997e-05 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860818+0000", 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6940000000000001e-05 2024-09-13T05:00:23.840 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34443 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799297+0000", 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.967899949, 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061720587, 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.841 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34443 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.842 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799297+0000", 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799301+0000", 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.5539999999999999e-06 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799312+0000", 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1044e-05 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.843 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799334+0000", 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.1708999999999998e-05 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860843+0000", 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061508588000000003 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.860935+0000", 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 9.2011999999999999e-05 2024-09-13T05:00:23.844 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861018+0000", 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.3337999999999994e-05 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34445 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799603+0000", 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.967593916, 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061552935000000003, 2024-09-13T05:00:23.845 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34445 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.846 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799603+0000", 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799602+0000", 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999986 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799609+0000", 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 6.6800000000000004e-06 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.847 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799625+0000", 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6401000000000001e-05 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799636+0000", 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1165000000000001e-05 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861058+0000", 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061422382999999997 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.848 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861109+0000", 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.0438000000000001e-05 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861156+0000", 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7377999999999999e-05 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.849 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34447 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799653+0000", 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.967544714, 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061623788999999998, 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34447 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.850 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799653+0000", 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799652+0000", 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.851 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799656+0000", 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.578e-06 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799681+0000", 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.4484000000000001e-05 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799692+0000", 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1618e-05 2024-09-13T05:00:23.852 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861185+0000", 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061492221999999999 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861231+0000", 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.6801999999999997e-05 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861276+0000", 2024-09-13T05:00:23.853 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.4960999999999999e-05 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34449 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799701+0000", 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.9674966, 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061706155999999998, 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.854 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34449 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799701+0000", 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.855 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799700+0000", 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.999999 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799704+0000", 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.3649999999999997e-06 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799716+0000", 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.149e-05 2024-09-13T05:00:23.856 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799743+0000", 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 2.6939000000000001e-05 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861306+0000", 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061563038 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861358+0000", 2024-09-13T05:00:23.857 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 5.2224e-05 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861407+0000", 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8822000000000001e-05 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34451 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799752+0000", 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.967445123, 2024-09-13T05:00:23.858 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061856368000000002, 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34451 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.859 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799752+0000", 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799751+0000", 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799755+0000", 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.0060000000000003e-06 2024-09-13T05:00:23.860 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799768+0000", 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2908000000000001e-05 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799780+0000", 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.1361e-05 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861439+0000", 2024-09-13T05:00:23.861 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061659380999999999 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861529+0000", 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 8.9611000000000007e-05 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861608+0000", 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 7.9790999999999996e-05 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.862 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "description": "osd_op(client.4394.0:34453 3.6 3:654c6808:::rbd_data.112ab86ccaf9.0000000000000005:8 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "initiated_at": "2024-09-13T04:58:23.799787+0000", 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "age": 119.967410042, 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061946465999999999, 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "type_data": { 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "flag_point": "started", 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_info": { 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client": "client.4394", 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "client_addr": "172.21.15.1:0/687156457", 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "tid": 34453 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "events": [ 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.863 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "initiated", 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "throttled", 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "header_read", 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799787+0000", 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4294967295.9999995 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.864 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "all_read", 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799803+0000", 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.6339000000000001e-05 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "dispatched", 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799817+0000", 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.4195999999999999e-05 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "queued_for_pg", 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.799830+0000", 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 1.2431e-05 2024-09-13T05:00:23.865 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "reached_pg", 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861638+0000", 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 0.061808729999999999 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "started", 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861686+0000", 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.8174000000000003e-05 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "event": "done", 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "time": "2024-09-13T04:58:23.861734+0000", 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: "duration": 4.7253999999999999e-05 2024-09-13T05:00:23.866 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-13T05:00:23.867 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-13T05:00:23.867 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:23.867 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:00:23.869 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:00:23.869 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:23.865+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:23.969 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-09-13T05:00:23.965+0000 7f9a814f8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26364) UID: 0 2024-09-13T05:00:23.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:00:24.032 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2024-09-13T05:00:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:00:24.052 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-13T05:00:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "ops": [], 2024-09-13T05:00:24.052 INFO:teuthology.orchestra.run.smithi144.stdout: "num_ops": 0 2024-09-13T05:00:24.052 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-13T05:00:24.053 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2024-09-13T05:00:24.053 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:00:24.067 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:00:24.070 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2024-09-13T05:00:24.070 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:00:24.090 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2024-09-13T05:00:24.090 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:00:24.098 INFO:teuthology.orchestra.run.smithi001.stdout:All operations completed A-OK! 2024-09-13T05:00:24.109 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-09-13T05:00:24.105+0000 7fb4e7505640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26362) UID: 0 2024-09-13T05:00:24.112 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2024-09-13T05:00:24.112 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:00:24.133 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2024-09-13T05:00:24.133 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:00:24.143 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:24.143 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-13T05:00:24.255 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T05:00:24.274 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T05:00:24.275 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2024-09-13T05:00:24.276 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:00:24.276 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2024-09-13T05:00:24.277 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:24.277 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:24.277 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:00:24.278 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:00:24.395 INFO:teuthology.orchestra.run.smithi144.stdout:ERROR: (22) Invalid argument 2024-09-13T05:00:24.395 INFO:teuthology.orchestra.run.smithi144.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:00:24.982 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T05:00:26.999 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T05:00:27.005 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:00:27.152 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a: {} 2024-09-13T05:00:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:00:27.165 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b: {} 2024-09-13T05:00:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:00:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c: {} 2024-09-13T05:00:27.176 INFO:teuthology.orchestra.run.smithi001.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:00:27.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T05:00:27.997 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2024-09-13T05:00:27.997 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T05:00:28.021 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:00:28.023 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T05:00:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2024-09-13T05:00:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T05:00:28.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T05:00:30.000 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T05:00:30.024 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T05:00:31.010 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T05:00:31.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T05:00:32.021 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T05:00:32.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T05:00:33.032 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T05:00:33.059 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T05:00:33.945 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T05:00:33.968 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T05:00:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T05:00:35.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T05:00:36.052 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T05:00:36.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T05:00:37.061 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T05:00:37.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T05:00:38.072 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T05:00:38.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T05:00:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T05:00:39.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T05:00:40.093 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T05:00:40.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T05:00:41.108 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T05:00:41.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T05:00:43.122 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T05:00:43.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-13T05:00:44.131 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T05:00:44.155 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T05:00:44.156 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T05:00:44.156 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T05:00:44.156 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T05:00:44.156 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T05:00:44.156 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:00:44.463 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:44.463 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":159,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T05:00:44.123806+0000","last_up_change":"2024-09-13T04:57:47.098606+0000","last_in_change":"2024-09-13T05:00:16.093048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T04:57:48.984599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T04:57:52.756781+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T04:58:02.446076+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":80,"snap_epoch":141,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6820","nonce":2346968854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6824","nonce":2346968854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6831","nonce":2346968854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6828","nonce":2346968854}]},"public_addr":"172.21.15.1:6820/2346968854","cluster_addr":"172.21.15.1:6824/2346968854","heartbeat_back_addr":"172.21.15.1:6831/2346968854","heartbeat_front_addr":"172.21.15.1:6828/2346968854","state":["exists","up"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6817","nonce":466532899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6821","nonce":466532899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6829","nonce":466532899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6825","nonce":466532899}]},"public_addr":"172.21.15.1:6817/466532899","cluster_addr":"172.21.15.1:6821/466532899","heartbeat_back_addr":"172.21.15.1:6829/466532899","heartbeat_front_addr":"172.21.15.1:6825/466532899","state":["exists","up"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6801","nonce":343712578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6803","nonce":343712578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6807","nonce":343712578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6805","nonce":343712578}]},"public_addr":"172.21.15.1:6801/343712578","cluster_addr":"172.21.15.1:6803/343712578","heartbeat_back_addr":"172.21.15.1:6807/343712578","heartbeat_front_addr":"172.21.15.1:6805/343712578","state":["exists","up"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6809","nonce":2021827348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6811","nonce":2021827348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6815","nonce":2021827348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6813","nonce":2021827348}]},"public_addr":"172.21.15.1:6809/2021827348","cluster_addr":"172.21.15.1:6811/2021827348","heartbeat_back_addr":"172.21.15.1:6815/2021827348","heartbeat_front_addr":"172.21.15.1:6813/2021827348","state":["exists","up"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6801","nonce":2183053562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6803","nonce":2183053562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6807","nonce":2183053562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6805","nonce":2183053562}]},"public_addr":"172.21.15.144:6801/2183053562","cluster_addr":"172.21.15.144:6803/2183053562","heartbeat_back_addr":"172.21.15.144:6807/2183053562","heartbeat_front_addr":"172.21.15.144:6805/2183053562","state":["exists","up"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6817","nonce":2765053006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6821","nonce":2765053006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6828","nonce":2765053006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6824","nonce":2765053006}]},"public_addr":"172.21.15.144:6817/2765053006","cluster_addr":"172.21.15.144:6821/2765053006","heartbeat_back_addr":"172.21.15.144:6828/2765053006","heartbeat_front_addr":"172.21.15.144:6824/2765053006","state":["exists","up"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6820","nonce":2721913793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6825","nonce":2721913793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6831","nonce":2721913793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6827","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6829","nonce":2721913793}]},"public_addr":"172.21.15.144:6820/2721913793","cluster_addr":"172.21.15.144:6825/2721913793","heartbeat_back_addr":"172.21.15.144:6831/2721913793","heartbeat_front_addr":"172.21.15.144:6829/2721913793","state":["exists","up"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6809","nonce":4202513770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6811","nonce":4202513770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6815","nonce":4202513770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6813","nonce":4202513770}]},"public_addr":"172.21.15.144:6809/4202513770","cluster_addr":"172.21.15.144:6811/4202513770","heartbeat_back_addr":"172.21.15.144:6815/4202513770","heartbeat_front_addr":"172.21.15.144:6813/4202513770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.092681+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.414226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.368701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.370707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.980085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.874193+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.219724+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.114987+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.6","osds":[6,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:00:44.483 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T05:00:44.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:00:44.782 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:44.783 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":159,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T05:00:44.123806+0000","last_up_change":"2024-09-13T04:57:47.098606+0000","last_in_change":"2024-09-13T05:00:16.093048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T04:57:48.984599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T04:57:52.756781+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T04:58:02.446076+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":80,"snap_epoch":141,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6820","nonce":2346968854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6824","nonce":2346968854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6831","nonce":2346968854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6828","nonce":2346968854}]},"public_addr":"172.21.15.1:6820/2346968854","cluster_addr":"172.21.15.1:6824/2346968854","heartbeat_back_addr":"172.21.15.1:6831/2346968854","heartbeat_front_addr":"172.21.15.1:6828/2346968854","state":["exists","up"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6817","nonce":466532899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6821","nonce":466532899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6829","nonce":466532899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6825","nonce":466532899}]},"public_addr":"172.21.15.1:6817/466532899","cluster_addr":"172.21.15.1:6821/466532899","heartbeat_back_addr":"172.21.15.1:6829/466532899","heartbeat_front_addr":"172.21.15.1:6825/466532899","state":["exists","up"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6801","nonce":343712578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6803","nonce":343712578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6807","nonce":343712578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6805","nonce":343712578}]},"public_addr":"172.21.15.1:6801/343712578","cluster_addr":"172.21.15.1:6803/343712578","heartbeat_back_addr":"172.21.15.1:6807/343712578","heartbeat_front_addr":"172.21.15.1:6805/343712578","state":["exists","up"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6809","nonce":2021827348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6811","nonce":2021827348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6815","nonce":2021827348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6813","nonce":2021827348}]},"public_addr":"172.21.15.1:6809/2021827348","cluster_addr":"172.21.15.1:6811/2021827348","heartbeat_back_addr":"172.21.15.1:6815/2021827348","heartbeat_front_addr":"172.21.15.1:6813/2021827348","state":["exists","up"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6801","nonce":2183053562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6803","nonce":2183053562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6807","nonce":2183053562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6805","nonce":2183053562}]},"public_addr":"172.21.15.144:6801/2183053562","cluster_addr":"172.21.15.144:6803/2183053562","heartbeat_back_addr":"172.21.15.144:6807/2183053562","heartbeat_front_addr":"172.21.15.144:6805/2183053562","state":["exists","up"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6817","nonce":2765053006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6821","nonce":2765053006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6828","nonce":2765053006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6824","nonce":2765053006}]},"public_addr":"172.21.15.144:6817/2765053006","cluster_addr":"172.21.15.144:6821/2765053006","heartbeat_back_addr":"172.21.15.144:6828/2765053006","heartbeat_front_addr":"172.21.15.144:6824/2765053006","state":["exists","up"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6820","nonce":2721913793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6825","nonce":2721913793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6831","nonce":2721913793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6827","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6829","nonce":2721913793}]},"public_addr":"172.21.15.144:6820/2721913793","cluster_addr":"172.21.15.144:6825/2721913793","heartbeat_back_addr":"172.21.15.144:6831/2721913793","heartbeat_front_addr":"172.21.15.144:6829/2721913793","state":["exists","up"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6809","nonce":4202513770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6811","nonce":4202513770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6815","nonce":4202513770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6813","nonce":4202513770}]},"public_addr":"172.21.15.144:6809/4202513770","cluster_addr":"172.21.15.144:6811/4202513770","heartbeat_back_addr":"172.21.15.144:6815/4202513770","heartbeat_front_addr":"172.21.15.144:6813/4202513770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.092681+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.414226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.368701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.370707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.980085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.874193+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.219724+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.114987+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.6","osds":[6,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:00:44.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T05:00:44.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T05:00:44.803 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T05:00:44.803 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T05:00:44.804 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T05:00:44.804 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T05:00:44.804 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T05:00:44.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T05:00:45.050 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:45.051 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T05:00:45.065 INFO:teuthology.orchestra.run.smithi001.stdout:60129542187 2024-09-13T05:00:45.065 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T05:00:45.067 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:45.067 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T05:00:45.078 INFO:teuthology.orchestra.run.smithi001.stdout:64424509480 2024-09-13T05:00:45.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T05:00:45.081 INFO:teuthology.orchestra.run.smithi001.stdout:64424509482 2024-09-13T05:00:45.082 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T05:00:45.083 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:45.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T05:00:45.084 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:45.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T05:00:45.094 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:45.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T05:00:45.590 INFO:teuthology.orchestra.run.smithi001.stdout:64424509478 2024-09-13T05:00:45.601 INFO:teuthology.orchestra.run.smithi001.stdout:64424509481 2024-09-13T05:00:45.605 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:45.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509478 for osd.1 2024-09-13T05:00:45.626 INFO:teuthology.orchestra.run.smithi001.stdout:64424509478 2024-09-13T05:00:45.626 INFO:teuthology.orchestra.run.smithi001.stdout:60129542186 2024-09-13T05:00:45.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509482 got 64424509481 for osd.7 2024-09-13T05:00:45.632 INFO:teuthology.orchestra.run.smithi001.stdout:64424509478 2024-09-13T05:00:45.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509479 for osd.4 2024-09-13T05:00:45.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542186 for osd.5 2024-09-13T05:00:45.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509478 for osd.3 2024-09-13T05:00:45.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509478 for osd.6 2024-09-13T05:00:45.658 INFO:teuthology.orchestra.run.smithi001.stdout:64424509478 2024-09-13T05:00:45.662 INFO:teuthology.orchestra.run.smithi001.stdout:64424509478 2024-09-13T05:00:45.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509478 for osd.0 2024-09-13T05:00:45.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509478 for osd.2 2024-09-13T05:00:46.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T05:00:46.633 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T05:00:46.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T05:00:46.654 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T05:00:46.656 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T05:00:46.657 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T05:00:46.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T05:00:46.681 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T05:00:47.160 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:47.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.1 2024-09-13T05:00:47.184 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:47.198 INFO:teuthology.orchestra.run.smithi001.stdout:60129542187 2024-09-13T05:00:47.204 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:47.211 INFO:teuthology.orchestra.run.smithi001.stdout:64424509480 2024-09-13T05:00:47.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542187 got 60129542187 for osd.5 2024-09-13T05:00:47.220 INFO:teuthology.orchestra.run.smithi001.stdout:64424509482 2024-09-13T05:00:47.220 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:47.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.6 2024-09-13T05:00:47.233 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:47.233 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:47.238 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:47.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509480 for osd.4 2024-09-13T05:00:47.243 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:47.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509482 got 64424509482 for osd.7 2024-09-13T05:00:47.250 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:47.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.2 2024-09-13T05:00:47.255 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:47.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.3 2024-09-13T05:00:47.262 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:47.263 INFO:teuthology.orchestra.run.smithi001.stdout:64424509479 2024-09-13T05:00:47.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509479 got 64424509479 for osd.0 2024-09-13T05:00:47.280 DEBUG:teuthology.parallel:result is None 2024-09-13T05:00:47.281 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T05:00:47.281 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:47.600 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:47.615 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-13T05:00:46.953201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18650,"ondisk_log_size":18650,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409076,"kb_used_data":14956,"kb_used_omap":12,"kb_used_meta":393971,"kb_avail":754565644,"statfs":{"total":773094113280,"available":772675219456,"internally_reserved":0,"allocated":15314944,"data_stored":8406678,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":403426873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.839381"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3926,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403598+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:43.403598+0000","last_peered":"2024-09-13T05:00:43.403598+0000","last_clean":"2024-09-13T05:00:43.403598+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:43.403598+0000","last_undegraded":"2024-09-13T05:00:43.403598+0000","last_fullsized":"2024-09-13T05:00:43.403598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:43.347419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"133'384","reported_seq":2267,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403397+0000","last_change":"2024-09-13T05:00:23.938537+0000","last_active":"2024-09-13T05:00:43.403397+0000","last_peered":"2024-09-13T05:00:43.403397+0000","last_clean":"2024-09-13T05:00:43.403397+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:43.403397+0000","last_undegraded":"2024-09-13T05:00:43.403397+0000","last_fullsized":"2024-09-13T05:00:43.403397+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3813,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402083+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:43.402083+0000","last_peered":"2024-09-13T05:00:43.402083+0000","last_clean":"2024-09-13T05:00:43.402083+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:43.402083+0000","last_undegraded":"2024-09-13T05:00:43.402083+0000","last_fullsized":"2024-09-13T05:00:43.402083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"141'604","reported_seq":2263,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402301+0000","last_change":"2024-09-13T05:00:23.938966+0000","last_active":"2024-09-13T05:00:43.402301+0000","last_peered":"2024-09-13T05:00:43.402301+0000","last_clean":"2024-09-13T05:00:43.402301+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:43.402301+0000","last_undegraded":"2024-09-13T05:00:43.402301+0000","last_fullsized":"2024-09-13T05:00:43.402301+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:19.948030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5236,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403387+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:43.403387+0000","last_peered":"2024-09-13T05:00:43.403387+0000","last_clean":"2024-09-13T05:00:43.403387+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:43.403387+0000","last_undegraded":"2024-09-13T05:00:43.403387+0000","last_fullsized":"2024-09-13T05:00:43.403387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"140'504","reported_seq":3990,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402018+0000","last_change":"2024-09-13T05:00:23.940610+0000","last_active":"2024-09-13T05:00:43.402018+0000","last_peered":"2024-09-13T05:00:43.402018+0000","last_clean":"2024-09-13T05:00:43.402018+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:00:43.402018+0000","last_undegraded":"2024-09-13T05:00:43.402018+0000","last_fullsized":"2024-09-13T05:00:43.402018+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:27:22.647601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"137'502","reported_seq":19474,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403105+0000","last_change":"2024-09-13T05:00:23.946784+0000","last_active":"2024-09-13T05:00:43.403105+0000","last_peered":"2024-09-13T05:00:43.403105+0000","last_clean":"2024-09-13T05:00:43.403105+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:00:43.403105+0000","last_undegraded":"2024-09-13T05:00:43.403105+0000","last_fullsized":"2024-09-13T05:00:43.403105+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.698921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"140'482","reported_seq":3078,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402754+0000","last_change":"2024-09-13T05:00:23.938257+0000","last_active":"2024-09-13T05:00:43.402754+0000","last_peered":"2024-09-13T05:00:43.402754+0000","last_clean":"2024-09-13T05:00:43.402754+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:43.402754+0000","last_undegraded":"2024-09-13T05:00:43.402754+0000","last_fullsized":"2024-09-13T05:00:43.402754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'332","reported_seq":3776,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403718+0000","last_change":"2024-09-13T05:00:23.939322+0000","last_active":"2024-09-13T05:00:43.403718+0000","last_peered":"2024-09-13T05:00:43.403718+0000","last_clean":"2024-09-13T05:00:43.403718+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:43.403718+0000","last_undegraded":"2024-09-13T05:00:43.403718+0000","last_fullsized":"2024-09-13T05:00:43.403718+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"140'560","reported_seq":2631,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122364+0000","last_change":"2024-09-13T05:00:23.939561+0000","last_active":"2024-09-13T05:00:43.122364+0000","last_peered":"2024-09-13T05:00:43.122364+0000","last_clean":"2024-09-13T05:00:43.122364+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:43.122364+0000","last_undegraded":"2024-09-13T05:00:43.122364+0000","last_fullsized":"2024-09-13T05:00:43.122364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"133'431","reported_seq":1280,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120694+0000","last_change":"2024-09-13T05:00:23.957788+0000","last_active":"2024-09-13T05:00:43.120694+0000","last_peered":"2024-09-13T05:00:43.120694+0000","last_clean":"2024-09-13T05:00:43.120694+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:43.120694+0000","last_undegraded":"2024-09-13T05:00:43.120694+0000","last_fullsized":"2024-09-13T05:00:43.120694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"136'744","reported_seq":4416,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402421+0000","last_change":"2024-09-13T05:00:24.076712+0000","last_active":"2024-09-13T05:00:43.402421+0000","last_peered":"2024-09-13T05:00:43.402421+0000","last_clean":"2024-09-13T05:00:43.402421+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:43.402421+0000","last_undegraded":"2024-09-13T05:00:43.402421+0000","last_fullsized":"2024-09-13T05:00:43.402421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401764+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:43.401764+0000","last_peered":"2024-09-13T05:00:43.401764+0000","last_clean":"2024-09-13T05:00:43.401764+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:43.401764+0000","last_undegraded":"2024-09-13T05:00:43.401764+0000","last_fullsized":"2024-09-13T05:00:43.401764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"141'339","reported_seq":2685,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119875+0000","last_change":"2024-09-13T05:00:23.995154+0000","last_active":"2024-09-13T05:00:43.119875+0000","last_peered":"2024-09-13T05:00:43.119875+0000","last_clean":"2024-09-13T05:00:43.119875+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:43.119875+0000","last_undegraded":"2024-09-13T05:00:43.119875+0000","last_fullsized":"2024-09-13T05:00:43.119875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402591+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:43.402591+0000","last_peered":"2024-09-13T05:00:43.402591+0000","last_clean":"2024-09-13T05:00:43.402591+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:43.402591+0000","last_undegraded":"2024-09-13T05:00:43.402591+0000","last_fullsized":"2024-09-13T05:00:43.402591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2239,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403580+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:43.403580+0000","last_peered":"2024-09-13T05:00:43.403580+0000","last_clean":"2024-09-13T05:00:43.403580+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:43.403580+0000","last_undegraded":"2024-09-13T05:00:43.403580+0000","last_fullsized":"2024-09-13T05:00:43.403580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402509+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:43.402509+0000","last_peered":"2024-09-13T05:00:43.402509+0000","last_clean":"2024-09-13T05:00:43.402509+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:43.402509+0000","last_undegraded":"2024-09-13T05:00:43.402509+0000","last_fullsized":"2024-09-13T05:00:43.402509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'633","reported_seq":1770,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119795+0000","last_change":"2024-09-13T05:00:42.129609+0000","last_active":"2024-09-13T05:00:43.119795+0000","last_peered":"2024-09-13T05:00:43.119795+0000","last_clean":"2024-09-13T05:00:43.119795+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:00:43.119795+0000","last_undegraded":"2024-09-13T05:00:43.119795+0000","last_fullsized":"2024-09-13T05:00:43.119795+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:35.254347+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403143+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:43.403143+0000","last_peered":"2024-09-13T05:00:43.403143+0000","last_clean":"2024-09-13T05:00:43.403143+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:43.403143+0000","last_undegraded":"2024-09-13T05:00:43.403143+0000","last_fullsized":"2024-09-13T05:00:43.403143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"135'558","reported_seq":2450,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122085+0000","last_change":"2024-09-13T05:00:23.939264+0000","last_active":"2024-09-13T05:00:43.122085+0000","last_peered":"2024-09-13T05:00:43.122085+0000","last_clean":"2024-09-13T05:00:43.122085+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:43.122085+0000","last_undegraded":"2024-09-13T05:00:43.122085+0000","last_fullsized":"2024-09-13T05:00:43.122085+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"141'925","reported_seq":4950,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120629+0000","last_change":"2024-09-13T05:00:23.954956+0000","last_active":"2024-09-13T05:00:43.120629+0000","last_peered":"2024-09-13T05:00:43.120629+0000","last_clean":"2024-09-13T05:00:43.120629+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:43.120629+0000","last_undegraded":"2024-09-13T05:00:43.120629+0000","last_fullsized":"2024-09-13T05:00:43.120629+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":155,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402441+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:43.402441+0000","last_peered":"2024-09-13T05:00:43.402441+0000","last_clean":"2024-09-13T05:00:43.402441+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:43.402441+0000","last_undegraded":"2024-09-13T05:00:43.402441+0000","last_fullsized":"2024-09-13T05:00:43.402441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"136'1217","reported_seq":2896,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402529+0000","last_change":"2024-09-13T05:00:23.937805+0000","last_active":"2024-09-13T05:00:43.402529+0000","last_peered":"2024-09-13T05:00:43.402529+0000","last_clean":"2024-09-13T05:00:43.402529+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:43.402529+0000","last_undegraded":"2024-09-13T05:00:43.402529+0000","last_fullsized":"2024-09-13T05:00:43.402529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-13T05:00:40.091300+0000","last_change":"2024-09-13T05:00:22.970392+0000","last_active":"2024-09-13T05:00:40.091300+0000","last_peered":"2024-09-13T05:00:40.091300+0000","last_clean":"2024-09-13T05:00:40.091300+0000","last_became_active":"2024-09-13T05:00:22.969656+0000","last_became_peered":"2024-09-13T05:00:22.969656+0000","last_unstale":"2024-09-13T05:00:40.091300+0000","last_undegraded":"2024-09-13T05:00:40.091300+0000","last_fullsized":"2024-09-13T05:00:40.091300+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:36.718927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"141'454","reported_seq":5161,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119546+0000","last_change":"2024-09-13T05:00:23.993994+0000","last_active":"2024-09-13T05:00:43.119546+0000","last_peered":"2024-09-13T05:00:43.119546+0000","last_clean":"2024-09-13T05:00:43.119546+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:43.119546+0000","last_undegraded":"2024-09-13T05:00:43.119546+0000","last_fullsized":"2024-09-13T05:00:43.119546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402085+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:43.402085+0000","last_peered":"2024-09-13T05:00:43.402085+0000","last_clean":"2024-09-13T05:00:43.402085+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:43.402085+0000","last_undegraded":"2024-09-13T05:00:43.402085+0000","last_fullsized":"2024-09-13T05:00:43.402085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"140'1034","reported_seq":3338,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403110+0000","last_change":"2024-09-13T05:00:23.945801+0000","last_active":"2024-09-13T05:00:43.403110+0000","last_peered":"2024-09-13T05:00:43.403110+0000","last_clean":"2024-09-13T05:00:43.403110+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:43.403110+0000","last_undegraded":"2024-09-13T05:00:43.403110+0000","last_fullsized":"2024-09-13T05:00:43.403110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":154,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402138+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:43.402138+0000","last_peered":"2024-09-13T05:00:43.402138+0000","last_clean":"2024-09-13T05:00:43.402138+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:43.402138+0000","last_undegraded":"2024-09-13T05:00:43.402138+0000","last_fullsized":"2024-09-13T05:00:43.402138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":198,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401751+0000","last_change":"2024-09-13T05:00:19.959464+0000","last_active":"2024-09-13T05:00:43.401751+0000","last_peered":"2024-09-13T05:00:43.401751+0000","last_clean":"2024-09-13T05:00:43.401751+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:43.401751+0000","last_undegraded":"2024-09-13T05:00:43.401751+0000","last_fullsized":"2024-09-13T05:00:43.401751+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:02.673583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"141'520","reported_seq":3953,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402840+0000","last_change":"2024-09-13T05:00:23.945803+0000","last_active":"2024-09-13T05:00:43.402840+0000","last_peered":"2024-09-13T05:00:43.402840+0000","last_clean":"2024-09-13T05:00:43.402840+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:43.402840+0000","last_undegraded":"2024-09-13T05:00:43.402840+0000","last_fullsized":"2024-09-13T05:00:43.402840+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"140'627","reported_seq":6114,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401843+0000","last_change":"2024-09-13T05:00:24.075211+0000","last_active":"2024-09-13T05:00:43.401843+0000","last_peered":"2024-09-13T05:00:43.401843+0000","last_clean":"2024-09-13T05:00:43.401843+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:43.401843+0000","last_undegraded":"2024-09-13T05:00:43.401843+0000","last_fullsized":"2024-09-13T05:00:43.401843+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"140'495","reported_seq":7321,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402176+0000","last_change":"2024-09-13T05:00:24.077894+0000","last_active":"2024-09-13T05:00:43.402176+0000","last_peered":"2024-09-13T05:00:43.402176+0000","last_clean":"2024-09-13T05:00:43.402176+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:00:43.402176+0000","last_undegraded":"2024-09-13T05:00:43.402176+0000","last_fullsized":"2024-09-13T05:00:43.402176+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:28:55.950141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"133'330","reported_seq":2147,"reported_epoch":159,"state":"peering","last_fresh":"2024-09-13T05:00:44.138442+0000","last_change":"2024-09-13T05:00:44.138442+0000","last_active":"2024-09-13T05:00:44.138442+0000","last_peered":"2024-09-13T05:00:21.072471+0000","last_clean":"2024-09-13T05:00:21.072471+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T05:00:44.138442+0000","last_undegraded":"2024-09-13T05:00:44.138442+0000","last_fullsized":"2024-09-13T05:00:44.138442+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:06.713133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"141'698","reported_seq":1939,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120411+0000","last_change":"2024-09-13T05:00:23.957946+0000","last_active":"2024-09-13T05:00:43.120411+0000","last_peered":"2024-09-13T05:00:43.120411+0000","last_clean":"2024-09-13T05:00:43.120411+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:43.120411+0000","last_undegraded":"2024-09-13T05:00:43.120411+0000","last_fullsized":"2024-09-13T05:00:43.120411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"141'472","reported_seq":5030,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402293+0000","last_change":"2024-09-13T05:00:24.078283+0000","last_active":"2024-09-13T05:00:43.402293+0000","last_peered":"2024-09-13T05:00:43.402293+0000","last_clean":"2024-09-13T05:00:43.402293+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:00:43.402293+0000","last_undegraded":"2024-09-13T05:00:43.402293+0000","last_fullsized":"2024-09-13T05:00:43.402293+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:11.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'451","reported_seq":2073,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122565+0000","last_change":"2024-09-13T05:00:23.939918+0000","last_active":"2024-09-13T05:00:43.122565+0000","last_peered":"2024-09-13T05:00:43.122565+0000","last_clean":"2024-09-13T05:00:43.122565+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:43.122565+0000","last_undegraded":"2024-09-13T05:00:43.122565+0000","last_fullsized":"2024-09-13T05:00:43.122565+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'855","reported_seq":8808,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.121268+0000","last_change":"2024-09-13T05:00:23.955180+0000","last_active":"2024-09-13T05:00:43.121268+0000","last_peered":"2024-09-13T05:00:43.121268+0000","last_clean":"2024-09-13T05:00:43.121268+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:00:43.121268+0000","last_undegraded":"2024-09-13T05:00:43.121268+0000","last_fullsized":"2024-09-13T05:00:43.121268+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:05.706484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"137'449","reported_seq":7169,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401613+0000","last_change":"2024-09-13T05:00:23.938922+0000","last_active":"2024-09-13T05:00:43.401613+0000","last_peered":"2024-09-13T05:00:43.401613+0000","last_clean":"2024-09-13T05:00:43.401613+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:43.401613+0000","last_undegraded":"2024-09-13T05:00:43.401613+0000","last_fullsized":"2024-09-13T05:00:43.401613+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:32:00.161910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"140'410","reported_seq":5181,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401745+0000","last_change":"2024-09-13T05:00:24.075984+0000","last_active":"2024-09-13T05:00:43.401745+0000","last_peered":"2024-09-13T05:00:43.401745+0000","last_clean":"2024-09-13T05:00:43.401745+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:43.401745+0000","last_undegraded":"2024-09-13T05:00:43.401745+0000","last_fullsized":"2024-09-13T05:00:43.401745+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'456","reported_seq":3703,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119633+0000","last_change":"2024-09-13T05:00:23.995986+0000","last_active":"2024-09-13T05:00:43.119633+0000","last_peered":"2024-09-13T05:00:43.119633+0000","last_clean":"2024-09-13T05:00:43.119633+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:43.119633+0000","last_undegraded":"2024-09-13T05:00:43.119633+0000","last_fullsized":"2024-09-13T05:00:43.119633+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"141'630","reported_seq":2061,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402040+0000","last_change":"2024-09-13T05:00:24.077226+0000","last_active":"2024-09-13T05:00:43.402040+0000","last_peered":"2024-09-13T05:00:43.402040+0000","last_clean":"2024-09-13T05:00:43.402040+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:43.402040+0000","last_undegraded":"2024-09-13T05:00:43.402040+0000","last_fullsized":"2024-09-13T05:00:43.402040+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45264,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326576,"statfs":{"total":96636764160,"available":96590413824,"internally_reserved":0,"allocated":1892352,"data_stored":899582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.099},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":1.016}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":1.212},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.282},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.143}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28892,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342948,"statfs":{"total":96636764160,"available":96607178752,"internally_reserved":0,"allocated":1908736,"data_stored":905443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.524}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.202},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":1.161},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":1.24}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.407}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":1.02},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.095},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.365}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.605},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.4399999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51028,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320812,"statfs":{"total":96636764160,"available":96584511488,"internally_reserved":0,"allocated":1904640,"data_stored":905424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.996}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":1.3109999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":1.282}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.196}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.161}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":1.3640000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.22}]}]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55116,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316724,"statfs":{"total":96636764160,"available":96580325376,"internally_reserved":0,"allocated":1896448,"data_stored":899601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.75},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.97099999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.25},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.1040000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.417}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.468}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.595},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.294}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59204,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312636,"statfs":{"total":96636764160,"available":96576139264,"internally_reserved":0,"allocated":1892352,"data_stored":899582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.80100000000000005}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":1.147},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.95799999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1739999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.306},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.82299999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59260,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312580,"statfs":{"total":96636764160,"available":96576081920,"internally_reserved":0,"allocated":1949696,"data_stored":1495811,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.317},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.075,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.503},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81599999999999995,"15min":0.82499999999999996},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1220000000000001,"5min":1.655,"15min":1.655},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.54300000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.60799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.1910000000000001,"5min":1.577,"15min":1.577},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.94599999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.738,"15min":1.738},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1000000000000001,"5min":1.766,"15min":1.766},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.93200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1904640,"data_stored":905424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.097},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":1.0620000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.82499999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":1.177},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2549999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1966080,"data_stored":1495811,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":1.171}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.145}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.97399999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.90800000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":1.204},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":1.319},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.601}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:47.619 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:47.924 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-13T05:00:46.953201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18650,"ondisk_log_size":18650,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409076,"kb_used_data":14956,"kb_used_omap":12,"kb_used_meta":393971,"kb_avail":754565644,"statfs":{"total":773094113280,"available":772675219456,"internally_reserved":0,"allocated":15314944,"data_stored":8406678,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":403426873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.839381"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3926,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403598+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:43.403598+0000","last_peered":"2024-09-13T05:00:43.403598+0000","last_clean":"2024-09-13T05:00:43.403598+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:43.403598+0000","last_undegraded":"2024-09-13T05:00:43.403598+0000","last_fullsized":"2024-09-13T05:00:43.403598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:43.347419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"133'384","reported_seq":2267,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403397+0000","last_change":"2024-09-13T05:00:23.938537+0000","last_active":"2024-09-13T05:00:43.403397+0000","last_peered":"2024-09-13T05:00:43.403397+0000","last_clean":"2024-09-13T05:00:43.403397+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:43.403397+0000","last_undegraded":"2024-09-13T05:00:43.403397+0000","last_fullsized":"2024-09-13T05:00:43.403397+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3813,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402083+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:43.402083+0000","last_peered":"2024-09-13T05:00:43.402083+0000","last_clean":"2024-09-13T05:00:43.402083+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:43.402083+0000","last_undegraded":"2024-09-13T05:00:43.402083+0000","last_fullsized":"2024-09-13T05:00:43.402083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"141'604","reported_seq":2263,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402301+0000","last_change":"2024-09-13T05:00:23.938966+0000","last_active":"2024-09-13T05:00:43.402301+0000","last_peered":"2024-09-13T05:00:43.402301+0000","last_clean":"2024-09-13T05:00:43.402301+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:43.402301+0000","last_undegraded":"2024-09-13T05:00:43.402301+0000","last_fullsized":"2024-09-13T05:00:43.402301+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:19.948030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5236,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403387+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:43.403387+0000","last_peered":"2024-09-13T05:00:43.403387+0000","last_clean":"2024-09-13T05:00:43.403387+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:43.403387+0000","last_undegraded":"2024-09-13T05:00:43.403387+0000","last_fullsized":"2024-09-13T05:00:43.403387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"140'504","reported_seq":3990,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402018+0000","last_change":"2024-09-13T05:00:23.940610+0000","last_active":"2024-09-13T05:00:43.402018+0000","last_peered":"2024-09-13T05:00:43.402018+0000","last_clean":"2024-09-13T05:00:43.402018+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:00:43.402018+0000","last_undegraded":"2024-09-13T05:00:43.402018+0000","last_fullsized":"2024-09-13T05:00:43.402018+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:27:22.647601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"137'502","reported_seq":19474,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403105+0000","last_change":"2024-09-13T05:00:23.946784+0000","last_active":"2024-09-13T05:00:43.403105+0000","last_peered":"2024-09-13T05:00:43.403105+0000","last_clean":"2024-09-13T05:00:43.403105+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:00:43.403105+0000","last_undegraded":"2024-09-13T05:00:43.403105+0000","last_fullsized":"2024-09-13T05:00:43.403105+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.698921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"140'482","reported_seq":3078,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402754+0000","last_change":"2024-09-13T05:00:23.938257+0000","last_active":"2024-09-13T05:00:43.402754+0000","last_peered":"2024-09-13T05:00:43.402754+0000","last_clean":"2024-09-13T05:00:43.402754+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:43.402754+0000","last_undegraded":"2024-09-13T05:00:43.402754+0000","last_fullsized":"2024-09-13T05:00:43.402754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'332","reported_seq":3776,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403718+0000","last_change":"2024-09-13T05:00:23.939322+0000","last_active":"2024-09-13T05:00:43.403718+0000","last_peered":"2024-09-13T05:00:43.403718+0000","last_clean":"2024-09-13T05:00:43.403718+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:43.403718+0000","last_undegraded":"2024-09-13T05:00:43.403718+0000","last_fullsized":"2024-09-13T05:00:43.403718+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"140'560","reported_seq":2631,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122364+0000","last_change":"2024-09-13T05:00:23.939561+0000","last_active":"2024-09-13T05:00:43.122364+0000","last_peered":"2024-09-13T05:00:43.122364+0000","last_clean":"2024-09-13T05:00:43.122364+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:43.122364+0000","last_undegraded":"2024-09-13T05:00:43.122364+0000","last_fullsized":"2024-09-13T05:00:43.122364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"133'431","reported_seq":1280,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120694+0000","last_change":"2024-09-13T05:00:23.957788+0000","last_active":"2024-09-13T05:00:43.120694+0000","last_peered":"2024-09-13T05:00:43.120694+0000","last_clean":"2024-09-13T05:00:43.120694+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:43.120694+0000","last_undegraded":"2024-09-13T05:00:43.120694+0000","last_fullsized":"2024-09-13T05:00:43.120694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"136'744","reported_seq":4416,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402421+0000","last_change":"2024-09-13T05:00:24.076712+0000","last_active":"2024-09-13T05:00:43.402421+0000","last_peered":"2024-09-13T05:00:43.402421+0000","last_clean":"2024-09-13T05:00:43.402421+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:43.402421+0000","last_undegraded":"2024-09-13T05:00:43.402421+0000","last_fullsized":"2024-09-13T05:00:43.402421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401764+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:43.401764+0000","last_peered":"2024-09-13T05:00:43.401764+0000","last_clean":"2024-09-13T05:00:43.401764+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:43.401764+0000","last_undegraded":"2024-09-13T05:00:43.401764+0000","last_fullsized":"2024-09-13T05:00:43.401764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"141'339","reported_seq":2685,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119875+0000","last_change":"2024-09-13T05:00:23.995154+0000","last_active":"2024-09-13T05:00:43.119875+0000","last_peered":"2024-09-13T05:00:43.119875+0000","last_clean":"2024-09-13T05:00:43.119875+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:43.119875+0000","last_undegraded":"2024-09-13T05:00:43.119875+0000","last_fullsized":"2024-09-13T05:00:43.119875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402591+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:43.402591+0000","last_peered":"2024-09-13T05:00:43.402591+0000","last_clean":"2024-09-13T05:00:43.402591+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:43.402591+0000","last_undegraded":"2024-09-13T05:00:43.402591+0000","last_fullsized":"2024-09-13T05:00:43.402591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2239,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403580+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:43.403580+0000","last_peered":"2024-09-13T05:00:43.403580+0000","last_clean":"2024-09-13T05:00:43.403580+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:43.403580+0000","last_undegraded":"2024-09-13T05:00:43.403580+0000","last_fullsized":"2024-09-13T05:00:43.403580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402509+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:43.402509+0000","last_peered":"2024-09-13T05:00:43.402509+0000","last_clean":"2024-09-13T05:00:43.402509+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:43.402509+0000","last_undegraded":"2024-09-13T05:00:43.402509+0000","last_fullsized":"2024-09-13T05:00:43.402509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'633","reported_seq":1770,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119795+0000","last_change":"2024-09-13T05:00:42.129609+0000","last_active":"2024-09-13T05:00:43.119795+0000","last_peered":"2024-09-13T05:00:43.119795+0000","last_clean":"2024-09-13T05:00:43.119795+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:00:43.119795+0000","last_undegraded":"2024-09-13T05:00:43.119795+0000","last_fullsized":"2024-09-13T05:00:43.119795+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:35.254347+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403143+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:43.403143+0000","last_peered":"2024-09-13T05:00:43.403143+0000","last_clean":"2024-09-13T05:00:43.403143+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:43.403143+0000","last_undegraded":"2024-09-13T05:00:43.403143+0000","last_fullsized":"2024-09-13T05:00:43.403143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"135'558","reported_seq":2450,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122085+0000","last_change":"2024-09-13T05:00:23.939264+0000","last_active":"2024-09-13T05:00:43.122085+0000","last_peered":"2024-09-13T05:00:43.122085+0000","last_clean":"2024-09-13T05:00:43.122085+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:43.122085+0000","last_undegraded":"2024-09-13T05:00:43.122085+0000","last_fullsized":"2024-09-13T05:00:43.122085+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"141'925","reported_seq":4950,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120629+0000","last_change":"2024-09-13T05:00:23.954956+0000","last_active":"2024-09-13T05:00:43.120629+0000","last_peered":"2024-09-13T05:00:43.120629+0000","last_clean":"2024-09-13T05:00:43.120629+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:43.120629+0000","last_undegraded":"2024-09-13T05:00:43.120629+0000","last_fullsized":"2024-09-13T05:00:43.120629+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":155,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402441+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:43.402441+0000","last_peered":"2024-09-13T05:00:43.402441+0000","last_clean":"2024-09-13T05:00:43.402441+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:43.402441+0000","last_undegraded":"2024-09-13T05:00:43.402441+0000","last_fullsized":"2024-09-13T05:00:43.402441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"136'1217","reported_seq":2896,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402529+0000","last_change":"2024-09-13T05:00:23.937805+0000","last_active":"2024-09-13T05:00:43.402529+0000","last_peered":"2024-09-13T05:00:43.402529+0000","last_clean":"2024-09-13T05:00:43.402529+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:43.402529+0000","last_undegraded":"2024-09-13T05:00:43.402529+0000","last_fullsized":"2024-09-13T05:00:43.402529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-13T05:00:40.091300+0000","last_change":"2024-09-13T05:00:22.970392+0000","last_active":"2024-09-13T05:00:40.091300+0000","last_peered":"2024-09-13T05:00:40.091300+0000","last_clean":"2024-09-13T05:00:40.091300+0000","last_became_active":"2024-09-13T05:00:22.969656+0000","last_became_peered":"2024-09-13T05:00:22.969656+0000","last_unstale":"2024-09-13T05:00:40.091300+0000","last_undegraded":"2024-09-13T05:00:40.091300+0000","last_fullsized":"2024-09-13T05:00:40.091300+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:36.718927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"141'454","reported_seq":5161,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119546+0000","last_change":"2024-09-13T05:00:23.993994+0000","last_active":"2024-09-13T05:00:43.119546+0000","last_peered":"2024-09-13T05:00:43.119546+0000","last_clean":"2024-09-13T05:00:43.119546+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:43.119546+0000","last_undegraded":"2024-09-13T05:00:43.119546+0000","last_fullsized":"2024-09-13T05:00:43.119546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402085+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:43.402085+0000","last_peered":"2024-09-13T05:00:43.402085+0000","last_clean":"2024-09-13T05:00:43.402085+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:43.402085+0000","last_undegraded":"2024-09-13T05:00:43.402085+0000","last_fullsized":"2024-09-13T05:00:43.402085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"140'1034","reported_seq":3338,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403110+0000","last_change":"2024-09-13T05:00:23.945801+0000","last_active":"2024-09-13T05:00:43.403110+0000","last_peered":"2024-09-13T05:00:43.403110+0000","last_clean":"2024-09-13T05:00:43.403110+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:43.403110+0000","last_undegraded":"2024-09-13T05:00:43.403110+0000","last_fullsized":"2024-09-13T05:00:43.403110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":154,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402138+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:43.402138+0000","last_peered":"2024-09-13T05:00:43.402138+0000","last_clean":"2024-09-13T05:00:43.402138+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:43.402138+0000","last_undegraded":"2024-09-13T05:00:43.402138+0000","last_fullsized":"2024-09-13T05:00:43.402138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":198,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401751+0000","last_change":"2024-09-13T05:00:19.959464+0000","last_active":"2024-09-13T05:00:43.401751+0000","last_peered":"2024-09-13T05:00:43.401751+0000","last_clean":"2024-09-13T05:00:43.401751+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:43.401751+0000","last_undegraded":"2024-09-13T05:00:43.401751+0000","last_fullsized":"2024-09-13T05:00:43.401751+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:02.673583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"141'520","reported_seq":3953,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402840+0000","last_change":"2024-09-13T05:00:23.945803+0000","last_active":"2024-09-13T05:00:43.402840+0000","last_peered":"2024-09-13T05:00:43.402840+0000","last_clean":"2024-09-13T05:00:43.402840+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:43.402840+0000","last_undegraded":"2024-09-13T05:00:43.402840+0000","last_fullsized":"2024-09-13T05:00:43.402840+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"140'627","reported_seq":6114,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401843+0000","last_change":"2024-09-13T05:00:24.075211+0000","last_active":"2024-09-13T05:00:43.401843+0000","last_peered":"2024-09-13T05:00:43.401843+0000","last_clean":"2024-09-13T05:00:43.401843+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:43.401843+0000","last_undegraded":"2024-09-13T05:00:43.401843+0000","last_fullsized":"2024-09-13T05:00:43.401843+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"140'495","reported_seq":7321,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402176+0000","last_change":"2024-09-13T05:00:24.077894+0000","last_active":"2024-09-13T05:00:43.402176+0000","last_peered":"2024-09-13T05:00:43.402176+0000","last_clean":"2024-09-13T05:00:43.402176+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:00:43.402176+0000","last_undegraded":"2024-09-13T05:00:43.402176+0000","last_fullsized":"2024-09-13T05:00:43.402176+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:28:55.950141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"133'330","reported_seq":2147,"reported_epoch":159,"state":"peering","last_fresh":"2024-09-13T05:00:44.138442+0000","last_change":"2024-09-13T05:00:44.138442+0000","last_active":"2024-09-13T05:00:44.138442+0000","last_peered":"2024-09-13T05:00:21.072471+0000","last_clean":"2024-09-13T05:00:21.072471+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T05:00:44.138442+0000","last_undegraded":"2024-09-13T05:00:44.138442+0000","last_fullsized":"2024-09-13T05:00:44.138442+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:06.713133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"141'698","reported_seq":1939,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120411+0000","last_change":"2024-09-13T05:00:23.957946+0000","last_active":"2024-09-13T05:00:43.120411+0000","last_peered":"2024-09-13T05:00:43.120411+0000","last_clean":"2024-09-13T05:00:43.120411+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:43.120411+0000","last_undegraded":"2024-09-13T05:00:43.120411+0000","last_fullsized":"2024-09-13T05:00:43.120411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"141'472","reported_seq":5030,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402293+0000","last_change":"2024-09-13T05:00:24.078283+0000","last_active":"2024-09-13T05:00:43.402293+0000","last_peered":"2024-09-13T05:00:43.402293+0000","last_clean":"2024-09-13T05:00:43.402293+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:00:43.402293+0000","last_undegraded":"2024-09-13T05:00:43.402293+0000","last_fullsized":"2024-09-13T05:00:43.402293+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:11.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'451","reported_seq":2073,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122565+0000","last_change":"2024-09-13T05:00:23.939918+0000","last_active":"2024-09-13T05:00:43.122565+0000","last_peered":"2024-09-13T05:00:43.122565+0000","last_clean":"2024-09-13T05:00:43.122565+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:43.122565+0000","last_undegraded":"2024-09-13T05:00:43.122565+0000","last_fullsized":"2024-09-13T05:00:43.122565+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'855","reported_seq":8808,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.121268+0000","last_change":"2024-09-13T05:00:23.955180+0000","last_active":"2024-09-13T05:00:43.121268+0000","last_peered":"2024-09-13T05:00:43.121268+0000","last_clean":"2024-09-13T05:00:43.121268+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:00:43.121268+0000","last_undegraded":"2024-09-13T05:00:43.121268+0000","last_fullsized":"2024-09-13T05:00:43.121268+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:05.706484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"137'449","reported_seq":7169,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401613+0000","last_change":"2024-09-13T05:00:23.938922+0000","last_active":"2024-09-13T05:00:43.401613+0000","last_peered":"2024-09-13T05:00:43.401613+0000","last_clean":"2024-09-13T05:00:43.401613+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:43.401613+0000","last_undegraded":"2024-09-13T05:00:43.401613+0000","last_fullsized":"2024-09-13T05:00:43.401613+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:32:00.161910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"140'410","reported_seq":5181,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401745+0000","last_change":"2024-09-13T05:00:24.075984+0000","last_active":"2024-09-13T05:00:43.401745+0000","last_peered":"2024-09-13T05:00:43.401745+0000","last_clean":"2024-09-13T05:00:43.401745+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:43.401745+0000","last_undegraded":"2024-09-13T05:00:43.401745+0000","last_fullsized":"2024-09-13T05:00:43.401745+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'456","reported_seq":3703,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119633+0000","last_change":"2024-09-13T05:00:23.995986+0000","last_active":"2024-09-13T05:00:43.119633+0000","last_peered":"2024-09-13T05:00:43.119633+0000","last_clean":"2024-09-13T05:00:43.119633+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:43.119633+0000","last_undegraded":"2024-09-13T05:00:43.119633+0000","last_fullsized":"2024-09-13T05:00:43.119633+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"141'630","reported_seq":2061,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402040+0000","last_change":"2024-09-13T05:00:24.077226+0000","last_active":"2024-09-13T05:00:43.402040+0000","last_peered":"2024-09-13T05:00:43.402040+0000","last_clean":"2024-09-13T05:00:43.402040+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:43.402040+0000","last_undegraded":"2024-09-13T05:00:43.402040+0000","last_fullsized":"2024-09-13T05:00:43.402040+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45264,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326576,"statfs":{"total":96636764160,"available":96590413824,"internally_reserved":0,"allocated":1892352,"data_stored":899582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.099},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":1.016}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":1.212},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.282},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.143}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28892,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342948,"statfs":{"total":96636764160,"available":96607178752,"internally_reserved":0,"allocated":1908736,"data_stored":905443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.524}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.202},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":1.161},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":1.24}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.407}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":1.02},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.095},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.365}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.605},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.4399999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51028,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320812,"statfs":{"total":96636764160,"available":96584511488,"internally_reserved":0,"allocated":1904640,"data_stored":905424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.996}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":1.3109999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":1.282}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.196}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.161}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":1.3640000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.22}]}]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55116,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316724,"statfs":{"total":96636764160,"available":96580325376,"internally_reserved":0,"allocated":1896448,"data_stored":899601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.75},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.97099999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.25},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.1040000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.417}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.468}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.595},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.294}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59204,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312636,"statfs":{"total":96636764160,"available":96576139264,"internally_reserved":0,"allocated":1892352,"data_stored":899582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.80100000000000005}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":1.147},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.95799999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1739999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.306},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.82299999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59260,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312580,"statfs":{"total":96636764160,"available":96576081920,"internally_reserved":0,"allocated":1949696,"data_stored":1495811,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.317},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.075,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.503},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81599999999999995,"15min":0.82499999999999996},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1220000000000001,"5min":1.655,"15min":1.655},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.54300000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.60799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.1910000000000001,"5min":1.577,"15min":1.577},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.94599999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.738,"15min":1.738},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1000000000000001,"5min":1.766,"15min":1.766},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.93200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1904640,"data_stored":905424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.097},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":1.0620000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.82499999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":1.177},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2549999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1966080,"data_stored":1495811,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":1.171}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.145}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.97399999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.90800000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":1.204},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":1.319},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.601}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:47.928 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T05:00:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:48.272 INFO:teuthology.orchestra.run.smithi001.stdout:{"fsid":"8850da77-4384-4335-9559-af2e805c4f72","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":194,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":160,"num_osds":8,"num_up_osds":8,"osd_up_since":1726203467,"num_in_osds":8,"osd_in_since":1726203616,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":40},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":501,"data_bytes":590406,"bytes_used":418893824,"bytes_avail":772675219456,"bytes_total":773094113280,"inactive_pgs_ratio":0.024390242993831635},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-13T04:58:48.921352+0000","services":{}},"progress_events":{}} 2024-09-13T05:00:48.289 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-13T05:00:48.290 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:48.585 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-13T05:00:46.953201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18650,"ondisk_log_size":18650,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409076,"kb_used_data":14956,"kb_used_omap":12,"kb_used_meta":393971,"kb_avail":754565644,"statfs":{"total":773094113280,"available":772675219456,"internally_reserved":0,"allocated":15314944,"data_stored":8406678,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":403426873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.839381"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3926,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403598+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:43.403598+0000","last_peered":"2024-09-13T05:00:43.403598+0000","last_clean":"2024-09-13T05:00:43.403598+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:43.403598+0000","last_undegraded":"2024-09-13T05:00:43.403598+0000","last_fullsized":"2024-09-13T05:00:43.403598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:43.347419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"133'384","reported_seq":2267,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403397+0000","last_change":"2024-09-13T05:00:23.938537+0000","last_active":"2024-09-13T05:00:43.403397+0000","last_peered":"2024-09-13T05:00:43.403397+0000","last_clean":"2024-09-13T05:00:43.403397+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:43.403397+0000","last_undegraded":"2024-09-13T05:00:43.403397+0000","last_fullsized":"2024-09-13T05:00:43.403397+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3813,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402083+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:43.402083+0000","last_peered":"2024-09-13T05:00:43.402083+0000","last_clean":"2024-09-13T05:00:43.402083+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:43.402083+0000","last_undegraded":"2024-09-13T05:00:43.402083+0000","last_fullsized":"2024-09-13T05:00:43.402083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"141'604","reported_seq":2263,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402301+0000","last_change":"2024-09-13T05:00:23.938966+0000","last_active":"2024-09-13T05:00:43.402301+0000","last_peered":"2024-09-13T05:00:43.402301+0000","last_clean":"2024-09-13T05:00:43.402301+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:43.402301+0000","last_undegraded":"2024-09-13T05:00:43.402301+0000","last_fullsized":"2024-09-13T05:00:43.402301+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:19.948030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5236,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403387+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:43.403387+0000","last_peered":"2024-09-13T05:00:43.403387+0000","last_clean":"2024-09-13T05:00:43.403387+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:43.403387+0000","last_undegraded":"2024-09-13T05:00:43.403387+0000","last_fullsized":"2024-09-13T05:00:43.403387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"140'504","reported_seq":3990,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402018+0000","last_change":"2024-09-13T05:00:23.940610+0000","last_active":"2024-09-13T05:00:43.402018+0000","last_peered":"2024-09-13T05:00:43.402018+0000","last_clean":"2024-09-13T05:00:43.402018+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:00:43.402018+0000","last_undegraded":"2024-09-13T05:00:43.402018+0000","last_fullsized":"2024-09-13T05:00:43.402018+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:27:22.647601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"137'502","reported_seq":19474,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403105+0000","last_change":"2024-09-13T05:00:23.946784+0000","last_active":"2024-09-13T05:00:43.403105+0000","last_peered":"2024-09-13T05:00:43.403105+0000","last_clean":"2024-09-13T05:00:43.403105+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:00:43.403105+0000","last_undegraded":"2024-09-13T05:00:43.403105+0000","last_fullsized":"2024-09-13T05:00:43.403105+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.698921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"140'482","reported_seq":3078,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402754+0000","last_change":"2024-09-13T05:00:23.938257+0000","last_active":"2024-09-13T05:00:43.402754+0000","last_peered":"2024-09-13T05:00:43.402754+0000","last_clean":"2024-09-13T05:00:43.402754+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:43.402754+0000","last_undegraded":"2024-09-13T05:00:43.402754+0000","last_fullsized":"2024-09-13T05:00:43.402754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'332","reported_seq":3776,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403718+0000","last_change":"2024-09-13T05:00:23.939322+0000","last_active":"2024-09-13T05:00:43.403718+0000","last_peered":"2024-09-13T05:00:43.403718+0000","last_clean":"2024-09-13T05:00:43.403718+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:43.403718+0000","last_undegraded":"2024-09-13T05:00:43.403718+0000","last_fullsized":"2024-09-13T05:00:43.403718+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"140'560","reported_seq":2631,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122364+0000","last_change":"2024-09-13T05:00:23.939561+0000","last_active":"2024-09-13T05:00:43.122364+0000","last_peered":"2024-09-13T05:00:43.122364+0000","last_clean":"2024-09-13T05:00:43.122364+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:43.122364+0000","last_undegraded":"2024-09-13T05:00:43.122364+0000","last_fullsized":"2024-09-13T05:00:43.122364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"133'431","reported_seq":1280,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120694+0000","last_change":"2024-09-13T05:00:23.957788+0000","last_active":"2024-09-13T05:00:43.120694+0000","last_peered":"2024-09-13T05:00:43.120694+0000","last_clean":"2024-09-13T05:00:43.120694+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:43.120694+0000","last_undegraded":"2024-09-13T05:00:43.120694+0000","last_fullsized":"2024-09-13T05:00:43.120694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"136'744","reported_seq":4416,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402421+0000","last_change":"2024-09-13T05:00:24.076712+0000","last_active":"2024-09-13T05:00:43.402421+0000","last_peered":"2024-09-13T05:00:43.402421+0000","last_clean":"2024-09-13T05:00:43.402421+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:43.402421+0000","last_undegraded":"2024-09-13T05:00:43.402421+0000","last_fullsized":"2024-09-13T05:00:43.402421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401764+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:43.401764+0000","last_peered":"2024-09-13T05:00:43.401764+0000","last_clean":"2024-09-13T05:00:43.401764+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:43.401764+0000","last_undegraded":"2024-09-13T05:00:43.401764+0000","last_fullsized":"2024-09-13T05:00:43.401764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"141'339","reported_seq":2685,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119875+0000","last_change":"2024-09-13T05:00:23.995154+0000","last_active":"2024-09-13T05:00:43.119875+0000","last_peered":"2024-09-13T05:00:43.119875+0000","last_clean":"2024-09-13T05:00:43.119875+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:43.119875+0000","last_undegraded":"2024-09-13T05:00:43.119875+0000","last_fullsized":"2024-09-13T05:00:43.119875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402591+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:43.402591+0000","last_peered":"2024-09-13T05:00:43.402591+0000","last_clean":"2024-09-13T05:00:43.402591+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:43.402591+0000","last_undegraded":"2024-09-13T05:00:43.402591+0000","last_fullsized":"2024-09-13T05:00:43.402591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2239,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403580+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:43.403580+0000","last_peered":"2024-09-13T05:00:43.403580+0000","last_clean":"2024-09-13T05:00:43.403580+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:43.403580+0000","last_undegraded":"2024-09-13T05:00:43.403580+0000","last_fullsized":"2024-09-13T05:00:43.403580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402509+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:43.402509+0000","last_peered":"2024-09-13T05:00:43.402509+0000","last_clean":"2024-09-13T05:00:43.402509+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:43.402509+0000","last_undegraded":"2024-09-13T05:00:43.402509+0000","last_fullsized":"2024-09-13T05:00:43.402509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'633","reported_seq":1770,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119795+0000","last_change":"2024-09-13T05:00:42.129609+0000","last_active":"2024-09-13T05:00:43.119795+0000","last_peered":"2024-09-13T05:00:43.119795+0000","last_clean":"2024-09-13T05:00:43.119795+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:00:43.119795+0000","last_undegraded":"2024-09-13T05:00:43.119795+0000","last_fullsized":"2024-09-13T05:00:43.119795+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:35.254347+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403143+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:43.403143+0000","last_peered":"2024-09-13T05:00:43.403143+0000","last_clean":"2024-09-13T05:00:43.403143+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:43.403143+0000","last_undegraded":"2024-09-13T05:00:43.403143+0000","last_fullsized":"2024-09-13T05:00:43.403143+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"135'558","reported_seq":2450,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122085+0000","last_change":"2024-09-13T05:00:23.939264+0000","last_active":"2024-09-13T05:00:43.122085+0000","last_peered":"2024-09-13T05:00:43.122085+0000","last_clean":"2024-09-13T05:00:43.122085+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:43.122085+0000","last_undegraded":"2024-09-13T05:00:43.122085+0000","last_fullsized":"2024-09-13T05:00:43.122085+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"141'925","reported_seq":4950,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120629+0000","last_change":"2024-09-13T05:00:23.954956+0000","last_active":"2024-09-13T05:00:43.120629+0000","last_peered":"2024-09-13T05:00:43.120629+0000","last_clean":"2024-09-13T05:00:43.120629+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:43.120629+0000","last_undegraded":"2024-09-13T05:00:43.120629+0000","last_fullsized":"2024-09-13T05:00:43.120629+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":155,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402441+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:43.402441+0000","last_peered":"2024-09-13T05:00:43.402441+0000","last_clean":"2024-09-13T05:00:43.402441+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:43.402441+0000","last_undegraded":"2024-09-13T05:00:43.402441+0000","last_fullsized":"2024-09-13T05:00:43.402441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"136'1217","reported_seq":2896,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402529+0000","last_change":"2024-09-13T05:00:23.937805+0000","last_active":"2024-09-13T05:00:43.402529+0000","last_peered":"2024-09-13T05:00:43.402529+0000","last_clean":"2024-09-13T05:00:43.402529+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:43.402529+0000","last_undegraded":"2024-09-13T05:00:43.402529+0000","last_fullsized":"2024-09-13T05:00:43.402529+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-13T05:00:40.091300+0000","last_change":"2024-09-13T05:00:22.970392+0000","last_active":"2024-09-13T05:00:40.091300+0000","last_peered":"2024-09-13T05:00:40.091300+0000","last_clean":"2024-09-13T05:00:40.091300+0000","last_became_active":"2024-09-13T05:00:22.969656+0000","last_became_peered":"2024-09-13T05:00:22.969656+0000","last_unstale":"2024-09-13T05:00:40.091300+0000","last_undegraded":"2024-09-13T05:00:40.091300+0000","last_fullsized":"2024-09-13T05:00:40.091300+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:36.718927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"141'454","reported_seq":5161,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119546+0000","last_change":"2024-09-13T05:00:23.993994+0000","last_active":"2024-09-13T05:00:43.119546+0000","last_peered":"2024-09-13T05:00:43.119546+0000","last_clean":"2024-09-13T05:00:43.119546+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:43.119546+0000","last_undegraded":"2024-09-13T05:00:43.119546+0000","last_fullsized":"2024-09-13T05:00:43.119546+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402085+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:43.402085+0000","last_peered":"2024-09-13T05:00:43.402085+0000","last_clean":"2024-09-13T05:00:43.402085+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:43.402085+0000","last_undegraded":"2024-09-13T05:00:43.402085+0000","last_fullsized":"2024-09-13T05:00:43.402085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"140'1034","reported_seq":3338,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.403110+0000","last_change":"2024-09-13T05:00:23.945801+0000","last_active":"2024-09-13T05:00:43.403110+0000","last_peered":"2024-09-13T05:00:43.403110+0000","last_clean":"2024-09-13T05:00:43.403110+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:43.403110+0000","last_undegraded":"2024-09-13T05:00:43.403110+0000","last_fullsized":"2024-09-13T05:00:43.403110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":154,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402138+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:43.402138+0000","last_peered":"2024-09-13T05:00:43.402138+0000","last_clean":"2024-09-13T05:00:43.402138+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:43.402138+0000","last_undegraded":"2024-09-13T05:00:43.402138+0000","last_fullsized":"2024-09-13T05:00:43.402138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":198,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401751+0000","last_change":"2024-09-13T05:00:19.959464+0000","last_active":"2024-09-13T05:00:43.401751+0000","last_peered":"2024-09-13T05:00:43.401751+0000","last_clean":"2024-09-13T05:00:43.401751+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:43.401751+0000","last_undegraded":"2024-09-13T05:00:43.401751+0000","last_fullsized":"2024-09-13T05:00:43.401751+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:02.673583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"141'520","reported_seq":3953,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402840+0000","last_change":"2024-09-13T05:00:23.945803+0000","last_active":"2024-09-13T05:00:43.402840+0000","last_peered":"2024-09-13T05:00:43.402840+0000","last_clean":"2024-09-13T05:00:43.402840+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:43.402840+0000","last_undegraded":"2024-09-13T05:00:43.402840+0000","last_fullsized":"2024-09-13T05:00:43.402840+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"140'627","reported_seq":6114,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401843+0000","last_change":"2024-09-13T05:00:24.075211+0000","last_active":"2024-09-13T05:00:43.401843+0000","last_peered":"2024-09-13T05:00:43.401843+0000","last_clean":"2024-09-13T05:00:43.401843+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:43.401843+0000","last_undegraded":"2024-09-13T05:00:43.401843+0000","last_fullsized":"2024-09-13T05:00:43.401843+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"140'495","reported_seq":7321,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402176+0000","last_change":"2024-09-13T05:00:24.077894+0000","last_active":"2024-09-13T05:00:43.402176+0000","last_peered":"2024-09-13T05:00:43.402176+0000","last_clean":"2024-09-13T05:00:43.402176+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:00:43.402176+0000","last_undegraded":"2024-09-13T05:00:43.402176+0000","last_fullsized":"2024-09-13T05:00:43.402176+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:28:55.950141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"133'330","reported_seq":2147,"reported_epoch":159,"state":"peering","last_fresh":"2024-09-13T05:00:44.138442+0000","last_change":"2024-09-13T05:00:44.138442+0000","last_active":"2024-09-13T05:00:44.138442+0000","last_peered":"2024-09-13T05:00:21.072471+0000","last_clean":"2024-09-13T05:00:21.072471+0000","last_became_active":"2024-09-13T04:58:03.927567+0000","last_became_peered":"2024-09-13T04:58:03.927567+0000","last_unstale":"2024-09-13T05:00:44.138442+0000","last_undegraded":"2024-09-13T05:00:44.138442+0000","last_fullsized":"2024-09-13T05:00:44.138442+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:06.713133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"141'698","reported_seq":1939,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.120411+0000","last_change":"2024-09-13T05:00:23.957946+0000","last_active":"2024-09-13T05:00:43.120411+0000","last_peered":"2024-09-13T05:00:43.120411+0000","last_clean":"2024-09-13T05:00:43.120411+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:43.120411+0000","last_undegraded":"2024-09-13T05:00:43.120411+0000","last_fullsized":"2024-09-13T05:00:43.120411+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"141'472","reported_seq":5030,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402293+0000","last_change":"2024-09-13T05:00:24.078283+0000","last_active":"2024-09-13T05:00:43.402293+0000","last_peered":"2024-09-13T05:00:43.402293+0000","last_clean":"2024-09-13T05:00:43.402293+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:00:43.402293+0000","last_undegraded":"2024-09-13T05:00:43.402293+0000","last_fullsized":"2024-09-13T05:00:43.402293+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:11.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'451","reported_seq":2073,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.122565+0000","last_change":"2024-09-13T05:00:23.939918+0000","last_active":"2024-09-13T05:00:43.122565+0000","last_peered":"2024-09-13T05:00:43.122565+0000","last_clean":"2024-09-13T05:00:43.122565+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:43.122565+0000","last_undegraded":"2024-09-13T05:00:43.122565+0000","last_fullsized":"2024-09-13T05:00:43.122565+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'855","reported_seq":8808,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.121268+0000","last_change":"2024-09-13T05:00:23.955180+0000","last_active":"2024-09-13T05:00:43.121268+0000","last_peered":"2024-09-13T05:00:43.121268+0000","last_clean":"2024-09-13T05:00:43.121268+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:00:43.121268+0000","last_undegraded":"2024-09-13T05:00:43.121268+0000","last_fullsized":"2024-09-13T05:00:43.121268+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:05.706484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"137'449","reported_seq":7169,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401613+0000","last_change":"2024-09-13T05:00:23.938922+0000","last_active":"2024-09-13T05:00:43.401613+0000","last_peered":"2024-09-13T05:00:43.401613+0000","last_clean":"2024-09-13T05:00:43.401613+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:43.401613+0000","last_undegraded":"2024-09-13T05:00:43.401613+0000","last_fullsized":"2024-09-13T05:00:43.401613+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:32:00.161910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"140'410","reported_seq":5181,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.401745+0000","last_change":"2024-09-13T05:00:24.075984+0000","last_active":"2024-09-13T05:00:43.401745+0000","last_peered":"2024-09-13T05:00:43.401745+0000","last_clean":"2024-09-13T05:00:43.401745+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:43.401745+0000","last_undegraded":"2024-09-13T05:00:43.401745+0000","last_fullsized":"2024-09-13T05:00:43.401745+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'456","reported_seq":3703,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.119633+0000","last_change":"2024-09-13T05:00:23.995986+0000","last_active":"2024-09-13T05:00:43.119633+0000","last_peered":"2024-09-13T05:00:43.119633+0000","last_clean":"2024-09-13T05:00:43.119633+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:43.119633+0000","last_undegraded":"2024-09-13T05:00:43.119633+0000","last_fullsized":"2024-09-13T05:00:43.119633+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"141'630","reported_seq":2061,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-13T05:00:43.402040+0000","last_change":"2024-09-13T05:00:24.077226+0000","last_active":"2024-09-13T05:00:43.402040+0000","last_peered":"2024-09-13T05:00:43.402040+0000","last_clean":"2024-09-13T05:00:43.402040+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:43.402040+0000","last_undegraded":"2024-09-13T05:00:43.402040+0000","last_fullsized":"2024-09-13T05:00:43.402040+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45264,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326576,"statfs":{"total":96636764160,"available":96590413824,"internally_reserved":0,"allocated":1892352,"data_stored":899582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.099},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":1.016}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":1.212},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.282},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.143}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28892,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342948,"statfs":{"total":96636764160,"available":96607178752,"internally_reserved":0,"allocated":1908736,"data_stored":905443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.524}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.202},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":1.161},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":1.24}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.407}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":1.02},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.095},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.365}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.605},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.4399999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51028,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320812,"statfs":{"total":96636764160,"available":96584511488,"internally_reserved":0,"allocated":1904640,"data_stored":905424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.85999999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.996}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":1.3109999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":1.282}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.196}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.161}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":1.3640000000000001}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.22}]}]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55116,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316724,"statfs":{"total":96636764160,"available":96580325376,"internally_reserved":0,"allocated":1896448,"data_stored":899601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.75},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.97099999999999997}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.25},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.1040000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.417}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.468}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.595},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.294}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59204,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312636,"statfs":{"total":96636764160,"available":96576139264,"internally_reserved":0,"allocated":1892352,"data_stored":899582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.80100000000000005}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":1.147},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.95799999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.1739999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.306},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.82299999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59260,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312580,"statfs":{"total":96636764160,"available":96576081920,"internally_reserved":0,"allocated":1949696,"data_stored":1495811,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.317},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.075,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.503},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81599999999999995,"15min":0.82499999999999996},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1220000000000001,"5min":1.655,"15min":1.655},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.54300000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.60799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.1910000000000001,"5min":1.577,"15min":1.577},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.94599999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.738,"15min":1.738},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1000000000000001,"5min":1.766,"15min":1.766},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.93200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1904640,"data_stored":905424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.097},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":1.0620000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.82499999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":1.177},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2549999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1966080,"data_stored":1495811,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":1.171}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.145}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.97399999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.90800000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":1.204},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":1.319},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.601}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:51.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:51.906 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:51.907 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:51.927 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-13T05:00:50.954317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18650,"ondisk_log_size":18650,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413288,"kb_used_data":15088,"kb_used_omap":12,"kb_used_meta":398067,"kb_avail":754561432,"statfs":{"total":773094113280,"available":772670906368,"internally_reserved":0,"allocated":15450112,"data_stored":8470940,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":407621177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.837845"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3927,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133058+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:44.133058+0000","last_peered":"2024-09-13T05:00:44.133058+0000","last_clean":"2024-09-13T05:00:44.133058+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:44.133058+0000","last_undegraded":"2024-09-13T05:00:44.133058+0000","last_fullsized":"2024-09-13T05:00:44.133058+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:43.347419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"133'384","reported_seq":2268,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132882+0000","last_change":"2024-09-13T05:00:23.938537+0000","last_active":"2024-09-13T05:00:44.132882+0000","last_peered":"2024-09-13T05:00:44.132882+0000","last_clean":"2024-09-13T05:00:44.132882+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:44.132882+0000","last_undegraded":"2024-09-13T05:00:44.132882+0000","last_fullsized":"2024-09-13T05:00:44.132882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3814,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130405+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:44.130405+0000","last_peered":"2024-09-13T05:00:44.130405+0000","last_clean":"2024-09-13T05:00:44.130405+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:44.130405+0000","last_undegraded":"2024-09-13T05:00:44.130405+0000","last_fullsized":"2024-09-13T05:00:44.130405+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"141'604","reported_seq":2264,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579388+0000","last_change":"2024-09-13T05:00:23.938966+0000","last_active":"2024-09-13T05:00:44.579388+0000","last_peered":"2024-09-13T05:00:44.579388+0000","last_clean":"2024-09-13T05:00:44.579388+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:44.579388+0000","last_undegraded":"2024-09-13T05:00:44.579388+0000","last_fullsized":"2024-09-13T05:00:44.579388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:19.948030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5237,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130640+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:44.130640+0000","last_peered":"2024-09-13T05:00:44.130640+0000","last_clean":"2024-09-13T05:00:44.130640+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:44.130640+0000","last_undegraded":"2024-09-13T05:00:44.130640+0000","last_fullsized":"2024-09-13T05:00:44.130640+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"140'504","reported_seq":3991,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579242+0000","last_change":"2024-09-13T05:00:23.940610+0000","last_active":"2024-09-13T05:00:44.579242+0000","last_peered":"2024-09-13T05:00:44.579242+0000","last_clean":"2024-09-13T05:00:44.579242+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:00:44.579242+0000","last_undegraded":"2024-09-13T05:00:44.579242+0000","last_fullsized":"2024-09-13T05:00:44.579242+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:27:22.647601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"137'502","reported_seq":19475,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130628+0000","last_change":"2024-09-13T05:00:23.946784+0000","last_active":"2024-09-13T05:00:44.130628+0000","last_peered":"2024-09-13T05:00:44.130628+0000","last_clean":"2024-09-13T05:00:44.130628+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:00:44.130628+0000","last_undegraded":"2024-09-13T05:00:44.130628+0000","last_fullsized":"2024-09-13T05:00:44.130628+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.698921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"140'482","reported_seq":3079,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132942+0000","last_change":"2024-09-13T05:00:23.938257+0000","last_active":"2024-09-13T05:00:44.132942+0000","last_peered":"2024-09-13T05:00:44.132942+0000","last_clean":"2024-09-13T05:00:44.132942+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:44.132942+0000","last_undegraded":"2024-09-13T05:00:44.132942+0000","last_fullsized":"2024-09-13T05:00:44.132942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'332","reported_seq":3777,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133187+0000","last_change":"2024-09-13T05:00:23.939322+0000","last_active":"2024-09-13T05:00:44.133187+0000","last_peered":"2024-09-13T05:00:44.133187+0000","last_clean":"2024-09-13T05:00:44.133187+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:44.133187+0000","last_undegraded":"2024-09-13T05:00:44.133187+0000","last_fullsized":"2024-09-13T05:00:44.133187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"140'560","reported_seq":2632,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577852+0000","last_change":"2024-09-13T05:00:23.939561+0000","last_active":"2024-09-13T05:00:44.577852+0000","last_peered":"2024-09-13T05:00:44.577852+0000","last_clean":"2024-09-13T05:00:44.577852+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:44.577852+0000","last_undegraded":"2024-09-13T05:00:44.577852+0000","last_fullsized":"2024-09-13T05:00:44.577852+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"133'431","reported_seq":1281,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578590+0000","last_change":"2024-09-13T05:00:23.957788+0000","last_active":"2024-09-13T05:00:44.578590+0000","last_peered":"2024-09-13T05:00:44.578590+0000","last_clean":"2024-09-13T05:00:44.578590+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:44.578590+0000","last_undegraded":"2024-09-13T05:00:44.578590+0000","last_fullsized":"2024-09-13T05:00:44.578590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"136'744","reported_seq":4417,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130067+0000","last_change":"2024-09-13T05:00:24.076712+0000","last_active":"2024-09-13T05:00:44.130067+0000","last_peered":"2024-09-13T05:00:44.130067+0000","last_clean":"2024-09-13T05:00:44.130067+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:44.130067+0000","last_undegraded":"2024-09-13T05:00:44.130067+0000","last_fullsized":"2024-09-13T05:00:44.130067+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130360+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:44.130360+0000","last_peered":"2024-09-13T05:00:44.130360+0000","last_clean":"2024-09-13T05:00:44.130360+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:44.130360+0000","last_undegraded":"2024-09-13T05:00:44.130360+0000","last_fullsized":"2024-09-13T05:00:44.130360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"141'339","reported_seq":2686,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143833+0000","last_change":"2024-09-13T05:00:23.995154+0000","last_active":"2024-09-13T05:00:44.143833+0000","last_peered":"2024-09-13T05:00:44.143833+0000","last_clean":"2024-09-13T05:00:44.143833+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:44.143833+0000","last_undegraded":"2024-09-13T05:00:44.143833+0000","last_fullsized":"2024-09-13T05:00:44.143833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.131058+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:44.131058+0000","last_peered":"2024-09-13T05:00:44.131058+0000","last_clean":"2024-09-13T05:00:44.131058+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:44.131058+0000","last_undegraded":"2024-09-13T05:00:44.131058+0000","last_fullsized":"2024-09-13T05:00:44.131058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2240,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130942+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:44.130942+0000","last_peered":"2024-09-13T05:00:44.130942+0000","last_clean":"2024-09-13T05:00:44.130942+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:44.130942+0000","last_undegraded":"2024-09-13T05:00:44.130942+0000","last_fullsized":"2024-09-13T05:00:44.130942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130598+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:44.130598+0000","last_peered":"2024-09-13T05:00:44.130598+0000","last_clean":"2024-09-13T05:00:44.130598+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:44.130598+0000","last_undegraded":"2024-09-13T05:00:44.130598+0000","last_fullsized":"2024-09-13T05:00:44.130598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'633","reported_seq":1771,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143707+0000","last_change":"2024-09-13T05:00:42.129609+0000","last_active":"2024-09-13T05:00:44.143707+0000","last_peered":"2024-09-13T05:00:44.143707+0000","last_clean":"2024-09-13T05:00:44.143707+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:00:44.143707+0000","last_undegraded":"2024-09-13T05:00:44.143707+0000","last_fullsized":"2024-09-13T05:00:44.143707+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:35.254347+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132888+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:44.132888+0000","last_peered":"2024-09-13T05:00:44.132888+0000","last_clean":"2024-09-13T05:00:44.132888+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:44.132888+0000","last_undegraded":"2024-09-13T05:00:44.132888+0000","last_fullsized":"2024-09-13T05:00:44.132888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"135'558","reported_seq":2451,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577992+0000","last_change":"2024-09-13T05:00:23.939264+0000","last_active":"2024-09-13T05:00:44.577992+0000","last_peered":"2024-09-13T05:00:44.577992+0000","last_clean":"2024-09-13T05:00:44.577992+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:44.577992+0000","last_undegraded":"2024-09-13T05:00:44.577992+0000","last_fullsized":"2024-09-13T05:00:44.577992+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"141'925","reported_seq":4951,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578241+0000","last_change":"2024-09-13T05:00:23.954956+0000","last_active":"2024-09-13T05:00:44.578241+0000","last_peered":"2024-09-13T05:00:44.578241+0000","last_clean":"2024-09-13T05:00:44.578241+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:44.578241+0000","last_undegraded":"2024-09-13T05:00:44.578241+0000","last_fullsized":"2024-09-13T05:00:44.578241+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":156,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130483+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:44.130483+0000","last_peered":"2024-09-13T05:00:44.130483+0000","last_clean":"2024-09-13T05:00:44.130483+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:44.130483+0000","last_undegraded":"2024-09-13T05:00:44.130483+0000","last_fullsized":"2024-09-13T05:00:44.130483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"136'1217","reported_seq":2897,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132642+0000","last_change":"2024-09-13T05:00:23.937805+0000","last_active":"2024-09-13T05:00:44.132642+0000","last_peered":"2024-09-13T05:00:44.132642+0000","last_clean":"2024-09-13T05:00:44.132642+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:44.132642+0000","last_undegraded":"2024-09-13T05:00:44.132642+0000","last_fullsized":"2024-09-13T05:00:44.132642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.153709+0000","last_change":"2024-09-13T05:00:45.153709+0000","last_active":"2024-09-13T05:00:45.153709+0000","last_peered":"2024-09-13T05:00:45.153709+0000","last_clean":"2024-09-13T05:00:45.153709+0000","last_became_active":"2024-09-13T05:00:45.152844+0000","last_became_peered":"2024-09-13T05:00:45.152844+0000","last_unstale":"2024-09-13T05:00:45.153709+0000","last_undegraded":"2024-09-13T05:00:45.153709+0000","last_fullsized":"2024-09-13T05:00:45.153709+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:00.253343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"141'454","reported_seq":5162,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143352+0000","last_change":"2024-09-13T05:00:23.993994+0000","last_active":"2024-09-13T05:00:44.143352+0000","last_peered":"2024-09-13T05:00:44.143352+0000","last_clean":"2024-09-13T05:00:44.143352+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:44.143352+0000","last_undegraded":"2024-09-13T05:00:44.143352+0000","last_fullsized":"2024-09-13T05:00:44.143352+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129930+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:44.129930+0000","last_peered":"2024-09-13T05:00:44.129930+0000","last_clean":"2024-09-13T05:00:44.129930+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:44.129930+0000","last_undegraded":"2024-09-13T05:00:44.129930+0000","last_fullsized":"2024-09-13T05:00:44.129930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"140'1034","reported_seq":3339,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130908+0000","last_change":"2024-09-13T05:00:23.945801+0000","last_active":"2024-09-13T05:00:44.130908+0000","last_peered":"2024-09-13T05:00:44.130908+0000","last_clean":"2024-09-13T05:00:44.130908+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:44.130908+0000","last_undegraded":"2024-09-13T05:00:44.130908+0000","last_fullsized":"2024-09-13T05:00:44.130908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":155,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130159+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:44.130159+0000","last_peered":"2024-09-13T05:00:44.130159+0000","last_clean":"2024-09-13T05:00:44.130159+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:44.130159+0000","last_undegraded":"2024-09-13T05:00:44.130159+0000","last_fullsized":"2024-09-13T05:00:44.130159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":199,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579202+0000","last_change":"2024-09-13T05:00:19.959464+0000","last_active":"2024-09-13T05:00:44.579202+0000","last_peered":"2024-09-13T05:00:44.579202+0000","last_clean":"2024-09-13T05:00:44.579202+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:44.579202+0000","last_undegraded":"2024-09-13T05:00:44.579202+0000","last_fullsized":"2024-09-13T05:00:44.579202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:02.673583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"141'520","reported_seq":3954,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130538+0000","last_change":"2024-09-13T05:00:23.945803+0000","last_active":"2024-09-13T05:00:44.130538+0000","last_peered":"2024-09-13T05:00:44.130538+0000","last_clean":"2024-09-13T05:00:44.130538+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:44.130538+0000","last_undegraded":"2024-09-13T05:00:44.130538+0000","last_fullsized":"2024-09-13T05:00:44.130538+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"140'627","reported_seq":6115,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129970+0000","last_change":"2024-09-13T05:00:24.075211+0000","last_active":"2024-09-13T05:00:44.129970+0000","last_peered":"2024-09-13T05:00:44.129970+0000","last_clean":"2024-09-13T05:00:44.129970+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:44.129970+0000","last_undegraded":"2024-09-13T05:00:44.129970+0000","last_fullsized":"2024-09-13T05:00:44.129970+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"140'495","reported_seq":7322,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130370+0000","last_change":"2024-09-13T05:00:24.077894+0000","last_active":"2024-09-13T05:00:44.130370+0000","last_peered":"2024-09-13T05:00:44.130370+0000","last_clean":"2024-09-13T05:00:44.130370+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:00:44.130370+0000","last_undegraded":"2024-09-13T05:00:44.130370+0000","last_fullsized":"2024-09-13T05:00:44.130370+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:28:55.950141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"133'330","reported_seq":2155,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.159707+0000","last_change":"2024-09-13T05:00:45.159707+0000","last_active":"2024-09-13T05:00:45.159707+0000","last_peered":"2024-09-13T05:00:45.159707+0000","last_clean":"2024-09-13T05:00:45.159707+0000","last_became_active":"2024-09-13T05:00:45.157021+0000","last_became_peered":"2024-09-13T05:00:45.157021+0000","last_unstale":"2024-09-13T05:00:45.159707+0000","last_undegraded":"2024-09-13T05:00:45.159707+0000","last_fullsized":"2024-09-13T05:00:45.159707+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:06.713133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"141'698","reported_seq":1940,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578444+0000","last_change":"2024-09-13T05:00:23.957946+0000","last_active":"2024-09-13T05:00:44.578444+0000","last_peered":"2024-09-13T05:00:44.578444+0000","last_clean":"2024-09-13T05:00:44.578444+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:44.578444+0000","last_undegraded":"2024-09-13T05:00:44.578444+0000","last_fullsized":"2024-09-13T05:00:44.578444+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"141'472","reported_seq":5031,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130630+0000","last_change":"2024-09-13T05:00:24.078283+0000","last_active":"2024-09-13T05:00:44.130630+0000","last_peered":"2024-09-13T05:00:44.130630+0000","last_clean":"2024-09-13T05:00:44.130630+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:00:44.130630+0000","last_undegraded":"2024-09-13T05:00:44.130630+0000","last_fullsized":"2024-09-13T05:00:44.130630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:11.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'451","reported_seq":2074,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577945+0000","last_change":"2024-09-13T05:00:23.939918+0000","last_active":"2024-09-13T05:00:44.577945+0000","last_peered":"2024-09-13T05:00:44.577945+0000","last_clean":"2024-09-13T05:00:44.577945+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:44.577945+0000","last_undegraded":"2024-09-13T05:00:44.577945+0000","last_fullsized":"2024-09-13T05:00:44.577945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'855","reported_seq":8809,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578593+0000","last_change":"2024-09-13T05:00:23.955180+0000","last_active":"2024-09-13T05:00:44.578593+0000","last_peered":"2024-09-13T05:00:44.578593+0000","last_clean":"2024-09-13T05:00:44.578593+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:00:44.578593+0000","last_undegraded":"2024-09-13T05:00:44.578593+0000","last_fullsized":"2024-09-13T05:00:44.578593+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:05.706484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"137'449","reported_seq":7170,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579189+0000","last_change":"2024-09-13T05:00:23.938922+0000","last_active":"2024-09-13T05:00:44.579189+0000","last_peered":"2024-09-13T05:00:44.579189+0000","last_clean":"2024-09-13T05:00:44.579189+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:44.579189+0000","last_undegraded":"2024-09-13T05:00:44.579189+0000","last_fullsized":"2024-09-13T05:00:44.579189+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:32:00.161910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"140'410","reported_seq":5182,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129937+0000","last_change":"2024-09-13T05:00:24.075984+0000","last_active":"2024-09-13T05:00:44.129937+0000","last_peered":"2024-09-13T05:00:44.129937+0000","last_clean":"2024-09-13T05:00:44.129937+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:44.129937+0000","last_undegraded":"2024-09-13T05:00:44.129937+0000","last_fullsized":"2024-09-13T05:00:44.129937+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'456","reported_seq":3704,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143364+0000","last_change":"2024-09-13T05:00:23.995986+0000","last_active":"2024-09-13T05:00:44.143364+0000","last_peered":"2024-09-13T05:00:44.143364+0000","last_clean":"2024-09-13T05:00:44.143364+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:44.143364+0000","last_undegraded":"2024-09-13T05:00:44.143364+0000","last_fullsized":"2024-09-13T05:00:44.143364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"141'630","reported_seq":2062,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130029+0000","last_change":"2024-09-13T05:00:24.077226+0000","last_active":"2024-09-13T05:00:44.130029+0000","last_peered":"2024-09-13T05:00:44.130029+0000","last_clean":"2024-09-13T05:00:44.130029+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:44.130029+0000","last_undegraded":"2024-09-13T05:00:44.130029+0000","last_fullsized":"2024-09-13T05:00:44.130029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45280,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326560,"statfs":{"total":96636764160,"available":96590397440,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.099},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":1.016}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":1.212},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.282},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.143}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.524}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.202},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":1.161},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":1.24}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.407}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":1.02},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.095},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.365}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.605},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.4399999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":3.972},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":0.95699999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":3.9009999999999998}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.083},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55136,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316704,"statfs":{"total":96636764160,"available":96580304896,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.83599999999999997}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.876}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2430000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.206}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.306},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2769999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59228,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312612,"statfs":{"total":96636764160,"available":96576114688,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.55100000000000005}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":1.8939999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.8420000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":1.9199999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":2.0430000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.9430000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.67400000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59276,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312564,"statfs":{"total":96636764160,"available":96576065536,"internally_reserved":0,"allocated":1961984,"data_stored":1501653,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.317},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.075,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.503},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81599999999999995,"15min":0.82499999999999996},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1220000000000001,"5min":1.655,"15min":1.655},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.54300000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.60799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.1910000000000001,"5min":1.577,"15min":1.577},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.94599999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.738,"15min":1.738},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1000000000000001,"5min":1.766,"15min":1.766},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.93200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.097},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":1.0620000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.82499999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":1.177},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2549999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51096,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320744,"statfs":{"total":96636764160,"available":96584441856,"internally_reserved":0,"allocated":1978368,"data_stored":1501653,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":1.171}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.145}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.97399999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.90800000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":1.204},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":1.319},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.601}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:51.931 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T05:00:51.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T05:00:52.249 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T05:00:52.259 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T05:00:52.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:52.547 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:52.548 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:52.564 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-13T05:00:50.954317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18650,"ondisk_log_size":18650,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413288,"kb_used_data":15088,"kb_used_omap":12,"kb_used_meta":398067,"kb_avail":754561432,"statfs":{"total":773094113280,"available":772670906368,"internally_reserved":0,"allocated":15450112,"data_stored":8470940,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":407621177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.837845"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3927,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133058+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:44.133058+0000","last_peered":"2024-09-13T05:00:44.133058+0000","last_clean":"2024-09-13T05:00:44.133058+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:44.133058+0000","last_undegraded":"2024-09-13T05:00:44.133058+0000","last_fullsized":"2024-09-13T05:00:44.133058+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:43.347419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"133'384","reported_seq":2268,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132882+0000","last_change":"2024-09-13T05:00:23.938537+0000","last_active":"2024-09-13T05:00:44.132882+0000","last_peered":"2024-09-13T05:00:44.132882+0000","last_clean":"2024-09-13T05:00:44.132882+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:44.132882+0000","last_undegraded":"2024-09-13T05:00:44.132882+0000","last_fullsized":"2024-09-13T05:00:44.132882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3814,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130405+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:44.130405+0000","last_peered":"2024-09-13T05:00:44.130405+0000","last_clean":"2024-09-13T05:00:44.130405+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:44.130405+0000","last_undegraded":"2024-09-13T05:00:44.130405+0000","last_fullsized":"2024-09-13T05:00:44.130405+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"141'604","reported_seq":2264,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579388+0000","last_change":"2024-09-13T05:00:23.938966+0000","last_active":"2024-09-13T05:00:44.579388+0000","last_peered":"2024-09-13T05:00:44.579388+0000","last_clean":"2024-09-13T05:00:44.579388+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:44.579388+0000","last_undegraded":"2024-09-13T05:00:44.579388+0000","last_fullsized":"2024-09-13T05:00:44.579388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:19.948030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5237,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130640+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:44.130640+0000","last_peered":"2024-09-13T05:00:44.130640+0000","last_clean":"2024-09-13T05:00:44.130640+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:44.130640+0000","last_undegraded":"2024-09-13T05:00:44.130640+0000","last_fullsized":"2024-09-13T05:00:44.130640+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"140'504","reported_seq":3991,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579242+0000","last_change":"2024-09-13T05:00:23.940610+0000","last_active":"2024-09-13T05:00:44.579242+0000","last_peered":"2024-09-13T05:00:44.579242+0000","last_clean":"2024-09-13T05:00:44.579242+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:00:44.579242+0000","last_undegraded":"2024-09-13T05:00:44.579242+0000","last_fullsized":"2024-09-13T05:00:44.579242+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:27:22.647601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"137'502","reported_seq":19475,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130628+0000","last_change":"2024-09-13T05:00:23.946784+0000","last_active":"2024-09-13T05:00:44.130628+0000","last_peered":"2024-09-13T05:00:44.130628+0000","last_clean":"2024-09-13T05:00:44.130628+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:00:44.130628+0000","last_undegraded":"2024-09-13T05:00:44.130628+0000","last_fullsized":"2024-09-13T05:00:44.130628+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.698921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"140'482","reported_seq":3079,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132942+0000","last_change":"2024-09-13T05:00:23.938257+0000","last_active":"2024-09-13T05:00:44.132942+0000","last_peered":"2024-09-13T05:00:44.132942+0000","last_clean":"2024-09-13T05:00:44.132942+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:44.132942+0000","last_undegraded":"2024-09-13T05:00:44.132942+0000","last_fullsized":"2024-09-13T05:00:44.132942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'332","reported_seq":3777,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133187+0000","last_change":"2024-09-13T05:00:23.939322+0000","last_active":"2024-09-13T05:00:44.133187+0000","last_peered":"2024-09-13T05:00:44.133187+0000","last_clean":"2024-09-13T05:00:44.133187+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:44.133187+0000","last_undegraded":"2024-09-13T05:00:44.133187+0000","last_fullsized":"2024-09-13T05:00:44.133187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"140'560","reported_seq":2632,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577852+0000","last_change":"2024-09-13T05:00:23.939561+0000","last_active":"2024-09-13T05:00:44.577852+0000","last_peered":"2024-09-13T05:00:44.577852+0000","last_clean":"2024-09-13T05:00:44.577852+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:44.577852+0000","last_undegraded":"2024-09-13T05:00:44.577852+0000","last_fullsized":"2024-09-13T05:00:44.577852+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"133'431","reported_seq":1281,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578590+0000","last_change":"2024-09-13T05:00:23.957788+0000","last_active":"2024-09-13T05:00:44.578590+0000","last_peered":"2024-09-13T05:00:44.578590+0000","last_clean":"2024-09-13T05:00:44.578590+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:44.578590+0000","last_undegraded":"2024-09-13T05:00:44.578590+0000","last_fullsized":"2024-09-13T05:00:44.578590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"136'744","reported_seq":4417,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130067+0000","last_change":"2024-09-13T05:00:24.076712+0000","last_active":"2024-09-13T05:00:44.130067+0000","last_peered":"2024-09-13T05:00:44.130067+0000","last_clean":"2024-09-13T05:00:44.130067+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:44.130067+0000","last_undegraded":"2024-09-13T05:00:44.130067+0000","last_fullsized":"2024-09-13T05:00:44.130067+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130360+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:44.130360+0000","last_peered":"2024-09-13T05:00:44.130360+0000","last_clean":"2024-09-13T05:00:44.130360+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:44.130360+0000","last_undegraded":"2024-09-13T05:00:44.130360+0000","last_fullsized":"2024-09-13T05:00:44.130360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"141'339","reported_seq":2686,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143833+0000","last_change":"2024-09-13T05:00:23.995154+0000","last_active":"2024-09-13T05:00:44.143833+0000","last_peered":"2024-09-13T05:00:44.143833+0000","last_clean":"2024-09-13T05:00:44.143833+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:44.143833+0000","last_undegraded":"2024-09-13T05:00:44.143833+0000","last_fullsized":"2024-09-13T05:00:44.143833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.131058+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:44.131058+0000","last_peered":"2024-09-13T05:00:44.131058+0000","last_clean":"2024-09-13T05:00:44.131058+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:44.131058+0000","last_undegraded":"2024-09-13T05:00:44.131058+0000","last_fullsized":"2024-09-13T05:00:44.131058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2240,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130942+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:44.130942+0000","last_peered":"2024-09-13T05:00:44.130942+0000","last_clean":"2024-09-13T05:00:44.130942+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:44.130942+0000","last_undegraded":"2024-09-13T05:00:44.130942+0000","last_fullsized":"2024-09-13T05:00:44.130942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130598+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:44.130598+0000","last_peered":"2024-09-13T05:00:44.130598+0000","last_clean":"2024-09-13T05:00:44.130598+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:44.130598+0000","last_undegraded":"2024-09-13T05:00:44.130598+0000","last_fullsized":"2024-09-13T05:00:44.130598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'633","reported_seq":1771,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143707+0000","last_change":"2024-09-13T05:00:42.129609+0000","last_active":"2024-09-13T05:00:44.143707+0000","last_peered":"2024-09-13T05:00:44.143707+0000","last_clean":"2024-09-13T05:00:44.143707+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:00:44.143707+0000","last_undegraded":"2024-09-13T05:00:44.143707+0000","last_fullsized":"2024-09-13T05:00:44.143707+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:35.254347+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132888+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:44.132888+0000","last_peered":"2024-09-13T05:00:44.132888+0000","last_clean":"2024-09-13T05:00:44.132888+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:44.132888+0000","last_undegraded":"2024-09-13T05:00:44.132888+0000","last_fullsized":"2024-09-13T05:00:44.132888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"135'558","reported_seq":2451,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577992+0000","last_change":"2024-09-13T05:00:23.939264+0000","last_active":"2024-09-13T05:00:44.577992+0000","last_peered":"2024-09-13T05:00:44.577992+0000","last_clean":"2024-09-13T05:00:44.577992+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:44.577992+0000","last_undegraded":"2024-09-13T05:00:44.577992+0000","last_fullsized":"2024-09-13T05:00:44.577992+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"141'925","reported_seq":4951,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578241+0000","last_change":"2024-09-13T05:00:23.954956+0000","last_active":"2024-09-13T05:00:44.578241+0000","last_peered":"2024-09-13T05:00:44.578241+0000","last_clean":"2024-09-13T05:00:44.578241+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:44.578241+0000","last_undegraded":"2024-09-13T05:00:44.578241+0000","last_fullsized":"2024-09-13T05:00:44.578241+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":156,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130483+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:44.130483+0000","last_peered":"2024-09-13T05:00:44.130483+0000","last_clean":"2024-09-13T05:00:44.130483+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:44.130483+0000","last_undegraded":"2024-09-13T05:00:44.130483+0000","last_fullsized":"2024-09-13T05:00:44.130483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"136'1217","reported_seq":2897,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132642+0000","last_change":"2024-09-13T05:00:23.937805+0000","last_active":"2024-09-13T05:00:44.132642+0000","last_peered":"2024-09-13T05:00:44.132642+0000","last_clean":"2024-09-13T05:00:44.132642+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:44.132642+0000","last_undegraded":"2024-09-13T05:00:44.132642+0000","last_fullsized":"2024-09-13T05:00:44.132642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.153709+0000","last_change":"2024-09-13T05:00:45.153709+0000","last_active":"2024-09-13T05:00:45.153709+0000","last_peered":"2024-09-13T05:00:45.153709+0000","last_clean":"2024-09-13T05:00:45.153709+0000","last_became_active":"2024-09-13T05:00:45.152844+0000","last_became_peered":"2024-09-13T05:00:45.152844+0000","last_unstale":"2024-09-13T05:00:45.153709+0000","last_undegraded":"2024-09-13T05:00:45.153709+0000","last_fullsized":"2024-09-13T05:00:45.153709+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:00.253343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"141'454","reported_seq":5162,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143352+0000","last_change":"2024-09-13T05:00:23.993994+0000","last_active":"2024-09-13T05:00:44.143352+0000","last_peered":"2024-09-13T05:00:44.143352+0000","last_clean":"2024-09-13T05:00:44.143352+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:44.143352+0000","last_undegraded":"2024-09-13T05:00:44.143352+0000","last_fullsized":"2024-09-13T05:00:44.143352+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129930+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:44.129930+0000","last_peered":"2024-09-13T05:00:44.129930+0000","last_clean":"2024-09-13T05:00:44.129930+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:44.129930+0000","last_undegraded":"2024-09-13T05:00:44.129930+0000","last_fullsized":"2024-09-13T05:00:44.129930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"140'1034","reported_seq":3339,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130908+0000","last_change":"2024-09-13T05:00:23.945801+0000","last_active":"2024-09-13T05:00:44.130908+0000","last_peered":"2024-09-13T05:00:44.130908+0000","last_clean":"2024-09-13T05:00:44.130908+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:44.130908+0000","last_undegraded":"2024-09-13T05:00:44.130908+0000","last_fullsized":"2024-09-13T05:00:44.130908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":155,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130159+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:44.130159+0000","last_peered":"2024-09-13T05:00:44.130159+0000","last_clean":"2024-09-13T05:00:44.130159+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:44.130159+0000","last_undegraded":"2024-09-13T05:00:44.130159+0000","last_fullsized":"2024-09-13T05:00:44.130159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":199,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579202+0000","last_change":"2024-09-13T05:00:19.959464+0000","last_active":"2024-09-13T05:00:44.579202+0000","last_peered":"2024-09-13T05:00:44.579202+0000","last_clean":"2024-09-13T05:00:44.579202+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:44.579202+0000","last_undegraded":"2024-09-13T05:00:44.579202+0000","last_fullsized":"2024-09-13T05:00:44.579202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:02.673583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"141'520","reported_seq":3954,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130538+0000","last_change":"2024-09-13T05:00:23.945803+0000","last_active":"2024-09-13T05:00:44.130538+0000","last_peered":"2024-09-13T05:00:44.130538+0000","last_clean":"2024-09-13T05:00:44.130538+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:44.130538+0000","last_undegraded":"2024-09-13T05:00:44.130538+0000","last_fullsized":"2024-09-13T05:00:44.130538+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"140'627","reported_seq":6115,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129970+0000","last_change":"2024-09-13T05:00:24.075211+0000","last_active":"2024-09-13T05:00:44.129970+0000","last_peered":"2024-09-13T05:00:44.129970+0000","last_clean":"2024-09-13T05:00:44.129970+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:44.129970+0000","last_undegraded":"2024-09-13T05:00:44.129970+0000","last_fullsized":"2024-09-13T05:00:44.129970+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"140'495","reported_seq":7322,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130370+0000","last_change":"2024-09-13T05:00:24.077894+0000","last_active":"2024-09-13T05:00:44.130370+0000","last_peered":"2024-09-13T05:00:44.130370+0000","last_clean":"2024-09-13T05:00:44.130370+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:00:44.130370+0000","last_undegraded":"2024-09-13T05:00:44.130370+0000","last_fullsized":"2024-09-13T05:00:44.130370+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:28:55.950141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"133'330","reported_seq":2155,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.159707+0000","last_change":"2024-09-13T05:00:45.159707+0000","last_active":"2024-09-13T05:00:45.159707+0000","last_peered":"2024-09-13T05:00:45.159707+0000","last_clean":"2024-09-13T05:00:45.159707+0000","last_became_active":"2024-09-13T05:00:45.157021+0000","last_became_peered":"2024-09-13T05:00:45.157021+0000","last_unstale":"2024-09-13T05:00:45.159707+0000","last_undegraded":"2024-09-13T05:00:45.159707+0000","last_fullsized":"2024-09-13T05:00:45.159707+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:06.713133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"141'698","reported_seq":1940,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578444+0000","last_change":"2024-09-13T05:00:23.957946+0000","last_active":"2024-09-13T05:00:44.578444+0000","last_peered":"2024-09-13T05:00:44.578444+0000","last_clean":"2024-09-13T05:00:44.578444+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:44.578444+0000","last_undegraded":"2024-09-13T05:00:44.578444+0000","last_fullsized":"2024-09-13T05:00:44.578444+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"141'472","reported_seq":5031,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130630+0000","last_change":"2024-09-13T05:00:24.078283+0000","last_active":"2024-09-13T05:00:44.130630+0000","last_peered":"2024-09-13T05:00:44.130630+0000","last_clean":"2024-09-13T05:00:44.130630+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:00:44.130630+0000","last_undegraded":"2024-09-13T05:00:44.130630+0000","last_fullsized":"2024-09-13T05:00:44.130630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:11.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'451","reported_seq":2074,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577945+0000","last_change":"2024-09-13T05:00:23.939918+0000","last_active":"2024-09-13T05:00:44.577945+0000","last_peered":"2024-09-13T05:00:44.577945+0000","last_clean":"2024-09-13T05:00:44.577945+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:44.577945+0000","last_undegraded":"2024-09-13T05:00:44.577945+0000","last_fullsized":"2024-09-13T05:00:44.577945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'855","reported_seq":8809,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578593+0000","last_change":"2024-09-13T05:00:23.955180+0000","last_active":"2024-09-13T05:00:44.578593+0000","last_peered":"2024-09-13T05:00:44.578593+0000","last_clean":"2024-09-13T05:00:44.578593+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:00:44.578593+0000","last_undegraded":"2024-09-13T05:00:44.578593+0000","last_fullsized":"2024-09-13T05:00:44.578593+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:05.706484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"137'449","reported_seq":7170,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579189+0000","last_change":"2024-09-13T05:00:23.938922+0000","last_active":"2024-09-13T05:00:44.579189+0000","last_peered":"2024-09-13T05:00:44.579189+0000","last_clean":"2024-09-13T05:00:44.579189+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:44.579189+0000","last_undegraded":"2024-09-13T05:00:44.579189+0000","last_fullsized":"2024-09-13T05:00:44.579189+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:32:00.161910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"140'410","reported_seq":5182,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129937+0000","last_change":"2024-09-13T05:00:24.075984+0000","last_active":"2024-09-13T05:00:44.129937+0000","last_peered":"2024-09-13T05:00:44.129937+0000","last_clean":"2024-09-13T05:00:44.129937+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:44.129937+0000","last_undegraded":"2024-09-13T05:00:44.129937+0000","last_fullsized":"2024-09-13T05:00:44.129937+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'456","reported_seq":3704,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143364+0000","last_change":"2024-09-13T05:00:23.995986+0000","last_active":"2024-09-13T05:00:44.143364+0000","last_peered":"2024-09-13T05:00:44.143364+0000","last_clean":"2024-09-13T05:00:44.143364+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:44.143364+0000","last_undegraded":"2024-09-13T05:00:44.143364+0000","last_fullsized":"2024-09-13T05:00:44.143364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"141'630","reported_seq":2062,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130029+0000","last_change":"2024-09-13T05:00:24.077226+0000","last_active":"2024-09-13T05:00:44.130029+0000","last_peered":"2024-09-13T05:00:44.130029+0000","last_clean":"2024-09-13T05:00:44.130029+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:44.130029+0000","last_undegraded":"2024-09-13T05:00:44.130029+0000","last_fullsized":"2024-09-13T05:00:44.130029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45280,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326560,"statfs":{"total":96636764160,"available":96590397440,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.099},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":1.016}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":1.212},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.282},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.143}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.524}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.202},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":1.161},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":1.24}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.407}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":1.02},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.095},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.365}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.605},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.4399999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":3.972},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":0.95699999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":3.9009999999999998}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.083},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55136,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316704,"statfs":{"total":96636764160,"available":96580304896,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.83599999999999997}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.876}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2430000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.206}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.306},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2769999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59228,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312612,"statfs":{"total":96636764160,"available":96576114688,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.55100000000000005}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":1.8939999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.8420000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":1.9199999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":2.0430000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.9430000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.67400000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59276,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312564,"statfs":{"total":96636764160,"available":96576065536,"internally_reserved":0,"allocated":1961984,"data_stored":1501653,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.317},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.075,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.503},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81599999999999995,"15min":0.82499999999999996},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1220000000000001,"5min":1.655,"15min":1.655},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.54300000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.60799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.1910000000000001,"5min":1.577,"15min":1.577},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.94599999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.738,"15min":1.738},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1000000000000001,"5min":1.766,"15min":1.766},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.93200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.097},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":1.0620000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.82499999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":1.177},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2549999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51096,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320744,"statfs":{"total":96636764160,"available":96584441856,"internally_reserved":0,"allocated":1978368,"data_stored":1501653,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":1.171}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.145}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.97399999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.90800000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":1.204},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":1.319},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.601}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:52.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:52.860 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:52.861 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:52.877 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-13T05:00:50.954317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18650,"ondisk_log_size":18650,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413288,"kb_used_data":15088,"kb_used_omap":12,"kb_used_meta":398067,"kb_avail":754561432,"statfs":{"total":773094113280,"available":772670906368,"internally_reserved":0,"allocated":15450112,"data_stored":8470940,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":407621177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.837845"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3927,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133058+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:44.133058+0000","last_peered":"2024-09-13T05:00:44.133058+0000","last_clean":"2024-09-13T05:00:44.133058+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:44.133058+0000","last_undegraded":"2024-09-13T05:00:44.133058+0000","last_fullsized":"2024-09-13T05:00:44.133058+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:43.347419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"133'384","reported_seq":2268,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132882+0000","last_change":"2024-09-13T05:00:23.938537+0000","last_active":"2024-09-13T05:00:44.132882+0000","last_peered":"2024-09-13T05:00:44.132882+0000","last_clean":"2024-09-13T05:00:44.132882+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:44.132882+0000","last_undegraded":"2024-09-13T05:00:44.132882+0000","last_fullsized":"2024-09-13T05:00:44.132882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3814,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130405+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:44.130405+0000","last_peered":"2024-09-13T05:00:44.130405+0000","last_clean":"2024-09-13T05:00:44.130405+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:44.130405+0000","last_undegraded":"2024-09-13T05:00:44.130405+0000","last_fullsized":"2024-09-13T05:00:44.130405+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"141'604","reported_seq":2264,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579388+0000","last_change":"2024-09-13T05:00:23.938966+0000","last_active":"2024-09-13T05:00:44.579388+0000","last_peered":"2024-09-13T05:00:44.579388+0000","last_clean":"2024-09-13T05:00:44.579388+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:44.579388+0000","last_undegraded":"2024-09-13T05:00:44.579388+0000","last_fullsized":"2024-09-13T05:00:44.579388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:19.948030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5237,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130640+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:44.130640+0000","last_peered":"2024-09-13T05:00:44.130640+0000","last_clean":"2024-09-13T05:00:44.130640+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:44.130640+0000","last_undegraded":"2024-09-13T05:00:44.130640+0000","last_fullsized":"2024-09-13T05:00:44.130640+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"140'504","reported_seq":3991,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579242+0000","last_change":"2024-09-13T05:00:23.940610+0000","last_active":"2024-09-13T05:00:44.579242+0000","last_peered":"2024-09-13T05:00:44.579242+0000","last_clean":"2024-09-13T05:00:44.579242+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:00:44.579242+0000","last_undegraded":"2024-09-13T05:00:44.579242+0000","last_fullsized":"2024-09-13T05:00:44.579242+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:27:22.647601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"137'502","reported_seq":19475,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130628+0000","last_change":"2024-09-13T05:00:23.946784+0000","last_active":"2024-09-13T05:00:44.130628+0000","last_peered":"2024-09-13T05:00:44.130628+0000","last_clean":"2024-09-13T05:00:44.130628+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:00:44.130628+0000","last_undegraded":"2024-09-13T05:00:44.130628+0000","last_fullsized":"2024-09-13T05:00:44.130628+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.698921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"140'482","reported_seq":3079,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132942+0000","last_change":"2024-09-13T05:00:23.938257+0000","last_active":"2024-09-13T05:00:44.132942+0000","last_peered":"2024-09-13T05:00:44.132942+0000","last_clean":"2024-09-13T05:00:44.132942+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:44.132942+0000","last_undegraded":"2024-09-13T05:00:44.132942+0000","last_fullsized":"2024-09-13T05:00:44.132942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'332","reported_seq":3777,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133187+0000","last_change":"2024-09-13T05:00:23.939322+0000","last_active":"2024-09-13T05:00:44.133187+0000","last_peered":"2024-09-13T05:00:44.133187+0000","last_clean":"2024-09-13T05:00:44.133187+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:44.133187+0000","last_undegraded":"2024-09-13T05:00:44.133187+0000","last_fullsized":"2024-09-13T05:00:44.133187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"140'560","reported_seq":2632,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577852+0000","last_change":"2024-09-13T05:00:23.939561+0000","last_active":"2024-09-13T05:00:44.577852+0000","last_peered":"2024-09-13T05:00:44.577852+0000","last_clean":"2024-09-13T05:00:44.577852+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:44.577852+0000","last_undegraded":"2024-09-13T05:00:44.577852+0000","last_fullsized":"2024-09-13T05:00:44.577852+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"133'431","reported_seq":1281,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578590+0000","last_change":"2024-09-13T05:00:23.957788+0000","last_active":"2024-09-13T05:00:44.578590+0000","last_peered":"2024-09-13T05:00:44.578590+0000","last_clean":"2024-09-13T05:00:44.578590+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:44.578590+0000","last_undegraded":"2024-09-13T05:00:44.578590+0000","last_fullsized":"2024-09-13T05:00:44.578590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"136'744","reported_seq":4417,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130067+0000","last_change":"2024-09-13T05:00:24.076712+0000","last_active":"2024-09-13T05:00:44.130067+0000","last_peered":"2024-09-13T05:00:44.130067+0000","last_clean":"2024-09-13T05:00:44.130067+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:44.130067+0000","last_undegraded":"2024-09-13T05:00:44.130067+0000","last_fullsized":"2024-09-13T05:00:44.130067+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130360+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:44.130360+0000","last_peered":"2024-09-13T05:00:44.130360+0000","last_clean":"2024-09-13T05:00:44.130360+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:44.130360+0000","last_undegraded":"2024-09-13T05:00:44.130360+0000","last_fullsized":"2024-09-13T05:00:44.130360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"141'339","reported_seq":2686,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143833+0000","last_change":"2024-09-13T05:00:23.995154+0000","last_active":"2024-09-13T05:00:44.143833+0000","last_peered":"2024-09-13T05:00:44.143833+0000","last_clean":"2024-09-13T05:00:44.143833+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:44.143833+0000","last_undegraded":"2024-09-13T05:00:44.143833+0000","last_fullsized":"2024-09-13T05:00:44.143833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.131058+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:44.131058+0000","last_peered":"2024-09-13T05:00:44.131058+0000","last_clean":"2024-09-13T05:00:44.131058+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:44.131058+0000","last_undegraded":"2024-09-13T05:00:44.131058+0000","last_fullsized":"2024-09-13T05:00:44.131058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2240,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130942+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:44.130942+0000","last_peered":"2024-09-13T05:00:44.130942+0000","last_clean":"2024-09-13T05:00:44.130942+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:44.130942+0000","last_undegraded":"2024-09-13T05:00:44.130942+0000","last_fullsized":"2024-09-13T05:00:44.130942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130598+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:44.130598+0000","last_peered":"2024-09-13T05:00:44.130598+0000","last_clean":"2024-09-13T05:00:44.130598+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:44.130598+0000","last_undegraded":"2024-09-13T05:00:44.130598+0000","last_fullsized":"2024-09-13T05:00:44.130598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'633","reported_seq":1771,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143707+0000","last_change":"2024-09-13T05:00:42.129609+0000","last_active":"2024-09-13T05:00:44.143707+0000","last_peered":"2024-09-13T05:00:44.143707+0000","last_clean":"2024-09-13T05:00:44.143707+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:00:44.143707+0000","last_undegraded":"2024-09-13T05:00:44.143707+0000","last_fullsized":"2024-09-13T05:00:44.143707+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:35.254347+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132888+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:44.132888+0000","last_peered":"2024-09-13T05:00:44.132888+0000","last_clean":"2024-09-13T05:00:44.132888+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:44.132888+0000","last_undegraded":"2024-09-13T05:00:44.132888+0000","last_fullsized":"2024-09-13T05:00:44.132888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"135'558","reported_seq":2451,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577992+0000","last_change":"2024-09-13T05:00:23.939264+0000","last_active":"2024-09-13T05:00:44.577992+0000","last_peered":"2024-09-13T05:00:44.577992+0000","last_clean":"2024-09-13T05:00:44.577992+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:44.577992+0000","last_undegraded":"2024-09-13T05:00:44.577992+0000","last_fullsized":"2024-09-13T05:00:44.577992+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"141'925","reported_seq":4951,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578241+0000","last_change":"2024-09-13T05:00:23.954956+0000","last_active":"2024-09-13T05:00:44.578241+0000","last_peered":"2024-09-13T05:00:44.578241+0000","last_clean":"2024-09-13T05:00:44.578241+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:44.578241+0000","last_undegraded":"2024-09-13T05:00:44.578241+0000","last_fullsized":"2024-09-13T05:00:44.578241+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":156,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130483+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:44.130483+0000","last_peered":"2024-09-13T05:00:44.130483+0000","last_clean":"2024-09-13T05:00:44.130483+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:44.130483+0000","last_undegraded":"2024-09-13T05:00:44.130483+0000","last_fullsized":"2024-09-13T05:00:44.130483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"136'1217","reported_seq":2897,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132642+0000","last_change":"2024-09-13T05:00:23.937805+0000","last_active":"2024-09-13T05:00:44.132642+0000","last_peered":"2024-09-13T05:00:44.132642+0000","last_clean":"2024-09-13T05:00:44.132642+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:44.132642+0000","last_undegraded":"2024-09-13T05:00:44.132642+0000","last_fullsized":"2024-09-13T05:00:44.132642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.153709+0000","last_change":"2024-09-13T05:00:45.153709+0000","last_active":"2024-09-13T05:00:45.153709+0000","last_peered":"2024-09-13T05:00:45.153709+0000","last_clean":"2024-09-13T05:00:45.153709+0000","last_became_active":"2024-09-13T05:00:45.152844+0000","last_became_peered":"2024-09-13T05:00:45.152844+0000","last_unstale":"2024-09-13T05:00:45.153709+0000","last_undegraded":"2024-09-13T05:00:45.153709+0000","last_fullsized":"2024-09-13T05:00:45.153709+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:00.253343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"141'454","reported_seq":5162,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143352+0000","last_change":"2024-09-13T05:00:23.993994+0000","last_active":"2024-09-13T05:00:44.143352+0000","last_peered":"2024-09-13T05:00:44.143352+0000","last_clean":"2024-09-13T05:00:44.143352+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:44.143352+0000","last_undegraded":"2024-09-13T05:00:44.143352+0000","last_fullsized":"2024-09-13T05:00:44.143352+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129930+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:44.129930+0000","last_peered":"2024-09-13T05:00:44.129930+0000","last_clean":"2024-09-13T05:00:44.129930+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:44.129930+0000","last_undegraded":"2024-09-13T05:00:44.129930+0000","last_fullsized":"2024-09-13T05:00:44.129930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"140'1034","reported_seq":3339,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130908+0000","last_change":"2024-09-13T05:00:23.945801+0000","last_active":"2024-09-13T05:00:44.130908+0000","last_peered":"2024-09-13T05:00:44.130908+0000","last_clean":"2024-09-13T05:00:44.130908+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:44.130908+0000","last_undegraded":"2024-09-13T05:00:44.130908+0000","last_fullsized":"2024-09-13T05:00:44.130908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":155,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130159+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:44.130159+0000","last_peered":"2024-09-13T05:00:44.130159+0000","last_clean":"2024-09-13T05:00:44.130159+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:44.130159+0000","last_undegraded":"2024-09-13T05:00:44.130159+0000","last_fullsized":"2024-09-13T05:00:44.130159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":199,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579202+0000","last_change":"2024-09-13T05:00:19.959464+0000","last_active":"2024-09-13T05:00:44.579202+0000","last_peered":"2024-09-13T05:00:44.579202+0000","last_clean":"2024-09-13T05:00:44.579202+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:44.579202+0000","last_undegraded":"2024-09-13T05:00:44.579202+0000","last_fullsized":"2024-09-13T05:00:44.579202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:02.673583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"141'520","reported_seq":3954,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130538+0000","last_change":"2024-09-13T05:00:23.945803+0000","last_active":"2024-09-13T05:00:44.130538+0000","last_peered":"2024-09-13T05:00:44.130538+0000","last_clean":"2024-09-13T05:00:44.130538+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:44.130538+0000","last_undegraded":"2024-09-13T05:00:44.130538+0000","last_fullsized":"2024-09-13T05:00:44.130538+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"140'627","reported_seq":6115,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129970+0000","last_change":"2024-09-13T05:00:24.075211+0000","last_active":"2024-09-13T05:00:44.129970+0000","last_peered":"2024-09-13T05:00:44.129970+0000","last_clean":"2024-09-13T05:00:44.129970+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:44.129970+0000","last_undegraded":"2024-09-13T05:00:44.129970+0000","last_fullsized":"2024-09-13T05:00:44.129970+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"140'495","reported_seq":7322,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130370+0000","last_change":"2024-09-13T05:00:24.077894+0000","last_active":"2024-09-13T05:00:44.130370+0000","last_peered":"2024-09-13T05:00:44.130370+0000","last_clean":"2024-09-13T05:00:44.130370+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:00:44.130370+0000","last_undegraded":"2024-09-13T05:00:44.130370+0000","last_fullsized":"2024-09-13T05:00:44.130370+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:28:55.950141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"133'330","reported_seq":2155,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.159707+0000","last_change":"2024-09-13T05:00:45.159707+0000","last_active":"2024-09-13T05:00:45.159707+0000","last_peered":"2024-09-13T05:00:45.159707+0000","last_clean":"2024-09-13T05:00:45.159707+0000","last_became_active":"2024-09-13T05:00:45.157021+0000","last_became_peered":"2024-09-13T05:00:45.157021+0000","last_unstale":"2024-09-13T05:00:45.159707+0000","last_undegraded":"2024-09-13T05:00:45.159707+0000","last_fullsized":"2024-09-13T05:00:45.159707+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:06.713133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"141'698","reported_seq":1940,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578444+0000","last_change":"2024-09-13T05:00:23.957946+0000","last_active":"2024-09-13T05:00:44.578444+0000","last_peered":"2024-09-13T05:00:44.578444+0000","last_clean":"2024-09-13T05:00:44.578444+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:44.578444+0000","last_undegraded":"2024-09-13T05:00:44.578444+0000","last_fullsized":"2024-09-13T05:00:44.578444+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"141'472","reported_seq":5031,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130630+0000","last_change":"2024-09-13T05:00:24.078283+0000","last_active":"2024-09-13T05:00:44.130630+0000","last_peered":"2024-09-13T05:00:44.130630+0000","last_clean":"2024-09-13T05:00:44.130630+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:00:44.130630+0000","last_undegraded":"2024-09-13T05:00:44.130630+0000","last_fullsized":"2024-09-13T05:00:44.130630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:11.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'451","reported_seq":2074,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577945+0000","last_change":"2024-09-13T05:00:23.939918+0000","last_active":"2024-09-13T05:00:44.577945+0000","last_peered":"2024-09-13T05:00:44.577945+0000","last_clean":"2024-09-13T05:00:44.577945+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:44.577945+0000","last_undegraded":"2024-09-13T05:00:44.577945+0000","last_fullsized":"2024-09-13T05:00:44.577945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'855","reported_seq":8809,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578593+0000","last_change":"2024-09-13T05:00:23.955180+0000","last_active":"2024-09-13T05:00:44.578593+0000","last_peered":"2024-09-13T05:00:44.578593+0000","last_clean":"2024-09-13T05:00:44.578593+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:00:44.578593+0000","last_undegraded":"2024-09-13T05:00:44.578593+0000","last_fullsized":"2024-09-13T05:00:44.578593+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:05.706484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"137'449","reported_seq":7170,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579189+0000","last_change":"2024-09-13T05:00:23.938922+0000","last_active":"2024-09-13T05:00:44.579189+0000","last_peered":"2024-09-13T05:00:44.579189+0000","last_clean":"2024-09-13T05:00:44.579189+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:44.579189+0000","last_undegraded":"2024-09-13T05:00:44.579189+0000","last_fullsized":"2024-09-13T05:00:44.579189+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:32:00.161910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"140'410","reported_seq":5182,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129937+0000","last_change":"2024-09-13T05:00:24.075984+0000","last_active":"2024-09-13T05:00:44.129937+0000","last_peered":"2024-09-13T05:00:44.129937+0000","last_clean":"2024-09-13T05:00:44.129937+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:44.129937+0000","last_undegraded":"2024-09-13T05:00:44.129937+0000","last_fullsized":"2024-09-13T05:00:44.129937+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'456","reported_seq":3704,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143364+0000","last_change":"2024-09-13T05:00:23.995986+0000","last_active":"2024-09-13T05:00:44.143364+0000","last_peered":"2024-09-13T05:00:44.143364+0000","last_clean":"2024-09-13T05:00:44.143364+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:44.143364+0000","last_undegraded":"2024-09-13T05:00:44.143364+0000","last_fullsized":"2024-09-13T05:00:44.143364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"141'630","reported_seq":2062,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130029+0000","last_change":"2024-09-13T05:00:24.077226+0000","last_active":"2024-09-13T05:00:44.130029+0000","last_peered":"2024-09-13T05:00:44.130029+0000","last_clean":"2024-09-13T05:00:44.130029+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:44.130029+0000","last_undegraded":"2024-09-13T05:00:44.130029+0000","last_fullsized":"2024-09-13T05:00:44.130029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45280,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326560,"statfs":{"total":96636764160,"available":96590397440,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.099},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":1.016}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":1.212},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.282},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.143}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.524}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.202},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":1.161},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":1.24}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.407}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":1.02},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.095},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.365}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.605},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.4399999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":3.972},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":0.95699999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":3.9009999999999998}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.083},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55136,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316704,"statfs":{"total":96636764160,"available":96580304896,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.83599999999999997}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.876}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2430000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.206}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.306},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2769999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59228,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312612,"statfs":{"total":96636764160,"available":96576114688,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.23000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.27300000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1339999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.55100000000000005}]},{"osd":1,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":1.8939999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.8420000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":1.9199999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83299999999999996,"15min":0.82299999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":2.0430000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92600000000000005,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3859999999999999,"5min":1.861,"15min":1.861},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.89100000000000001,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.464,"5min":1.627,"15min":1.627},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92200000000000004,"15min":0.91800000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":1.9430000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5740000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95599999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.454,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0049999999999999,"15min":1.02},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5660000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.99399999999999999,"15min":1.012},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.8100000000000001,"5min":2.254,"15min":2.254},"last":0.67400000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59276,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312564,"statfs":{"total":96636764160,"available":96576065536,"internally_reserved":0,"allocated":1961984,"data_stored":1501653,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.317},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.075,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.503},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81599999999999995,"15min":0.82499999999999996},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1220000000000001,"5min":1.655,"15min":1.655},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.54300000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.60799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.1910000000000001,"5min":1.577,"15min":1.577},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.94599999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.738,"15min":1.738},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1000000000000001,"5min":1.766,"15min":1.766},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.93200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.097},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":1.0620000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.753},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":0.82499999999999996}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":1.177},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.2549999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51096,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320744,"statfs":{"total":96636764160,"available":96584441856,"internally_reserved":0,"allocated":1978368,"data_stored":1501653,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.91600000000000004},"min":{"1min":0.47599999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.609,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0920000000000001,"5min":1.129,"15min":1.129},"last":1.171}]},{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.753,"15min":0.73299999999999998},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2070000000000001,"5min":1.23,"15min":1.23},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.145}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.55900000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.97999999999999998,"15min":1.006},"min":{"1min":0.51600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.7,"5min":4.633,"15min":4.633},"last":0.97399999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.86299999999999999,"15min":0.84599999999999997},"min":{"1min":0.70399999999999996,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.83699999999999997},"min":{"1min":0.53300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.90800000000000003}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.768,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":1.204},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.85199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.304,"5min":1.409,"15min":1.409},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.91900000000000004,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0250000000000004,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":1.319},{"interface":"front","average":{"1min":1.113,"5min":1.1180000000000001,"15min":1.119},"min":{"1min":0.499,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.0640000000000001,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.601}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.94599999999999995,"15min":0.93799999999999994},"min":{"1min":0.47299999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.93000000000000005,"15min":0.90600000000000003},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:52.880 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T05:00:52.880 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:53.179 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:53.180 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:53.197 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-13T05:00:52.954767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18650,"ondisk_log_size":18650,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413288,"kb_used_data":15088,"kb_used_omap":12,"kb_used_meta":398067,"kb_avail":754561432,"statfs":{"total":773094113280,"available":772670906368,"internally_reserved":0,"allocated":15450112,"data_stored":8470940,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":407621177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.827817"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3927,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133058+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:44.133058+0000","last_peered":"2024-09-13T05:00:44.133058+0000","last_clean":"2024-09-13T05:00:44.133058+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:44.133058+0000","last_undegraded":"2024-09-13T05:00:44.133058+0000","last_fullsized":"2024-09-13T05:00:44.133058+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:43.347419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"133'384","reported_seq":2268,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132882+0000","last_change":"2024-09-13T05:00:23.938537+0000","last_active":"2024-09-13T05:00:44.132882+0000","last_peered":"2024-09-13T05:00:44.132882+0000","last_clean":"2024-09-13T05:00:44.132882+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:44.132882+0000","last_undegraded":"2024-09-13T05:00:44.132882+0000","last_fullsized":"2024-09-13T05:00:44.132882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3814,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130405+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:44.130405+0000","last_peered":"2024-09-13T05:00:44.130405+0000","last_clean":"2024-09-13T05:00:44.130405+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:44.130405+0000","last_undegraded":"2024-09-13T05:00:44.130405+0000","last_fullsized":"2024-09-13T05:00:44.130405+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"141'604","reported_seq":2264,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579388+0000","last_change":"2024-09-13T05:00:23.938966+0000","last_active":"2024-09-13T05:00:44.579388+0000","last_peered":"2024-09-13T05:00:44.579388+0000","last_clean":"2024-09-13T05:00:44.579388+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:44.579388+0000","last_undegraded":"2024-09-13T05:00:44.579388+0000","last_fullsized":"2024-09-13T05:00:44.579388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:19.948030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5237,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130640+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:44.130640+0000","last_peered":"2024-09-13T05:00:44.130640+0000","last_clean":"2024-09-13T05:00:44.130640+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:44.130640+0000","last_undegraded":"2024-09-13T05:00:44.130640+0000","last_fullsized":"2024-09-13T05:00:44.130640+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"140'504","reported_seq":3991,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579242+0000","last_change":"2024-09-13T05:00:23.940610+0000","last_active":"2024-09-13T05:00:44.579242+0000","last_peered":"2024-09-13T05:00:44.579242+0000","last_clean":"2024-09-13T05:00:44.579242+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:00:44.579242+0000","last_undegraded":"2024-09-13T05:00:44.579242+0000","last_fullsized":"2024-09-13T05:00:44.579242+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:27:22.647601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"137'502","reported_seq":19475,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130628+0000","last_change":"2024-09-13T05:00:23.946784+0000","last_active":"2024-09-13T05:00:44.130628+0000","last_peered":"2024-09-13T05:00:44.130628+0000","last_clean":"2024-09-13T05:00:44.130628+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:00:44.130628+0000","last_undegraded":"2024-09-13T05:00:44.130628+0000","last_fullsized":"2024-09-13T05:00:44.130628+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.698921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"140'482","reported_seq":3079,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132942+0000","last_change":"2024-09-13T05:00:23.938257+0000","last_active":"2024-09-13T05:00:44.132942+0000","last_peered":"2024-09-13T05:00:44.132942+0000","last_clean":"2024-09-13T05:00:44.132942+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:44.132942+0000","last_undegraded":"2024-09-13T05:00:44.132942+0000","last_fullsized":"2024-09-13T05:00:44.132942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'332","reported_seq":3777,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133187+0000","last_change":"2024-09-13T05:00:23.939322+0000","last_active":"2024-09-13T05:00:44.133187+0000","last_peered":"2024-09-13T05:00:44.133187+0000","last_clean":"2024-09-13T05:00:44.133187+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:44.133187+0000","last_undegraded":"2024-09-13T05:00:44.133187+0000","last_fullsized":"2024-09-13T05:00:44.133187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"140'560","reported_seq":2632,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577852+0000","last_change":"2024-09-13T05:00:23.939561+0000","last_active":"2024-09-13T05:00:44.577852+0000","last_peered":"2024-09-13T05:00:44.577852+0000","last_clean":"2024-09-13T05:00:44.577852+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:44.577852+0000","last_undegraded":"2024-09-13T05:00:44.577852+0000","last_fullsized":"2024-09-13T05:00:44.577852+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"133'431","reported_seq":1281,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578590+0000","last_change":"2024-09-13T05:00:23.957788+0000","last_active":"2024-09-13T05:00:44.578590+0000","last_peered":"2024-09-13T05:00:44.578590+0000","last_clean":"2024-09-13T05:00:44.578590+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:44.578590+0000","last_undegraded":"2024-09-13T05:00:44.578590+0000","last_fullsized":"2024-09-13T05:00:44.578590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"136'744","reported_seq":4417,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130067+0000","last_change":"2024-09-13T05:00:24.076712+0000","last_active":"2024-09-13T05:00:44.130067+0000","last_peered":"2024-09-13T05:00:44.130067+0000","last_clean":"2024-09-13T05:00:44.130067+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:44.130067+0000","last_undegraded":"2024-09-13T05:00:44.130067+0000","last_fullsized":"2024-09-13T05:00:44.130067+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130360+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:44.130360+0000","last_peered":"2024-09-13T05:00:44.130360+0000","last_clean":"2024-09-13T05:00:44.130360+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:44.130360+0000","last_undegraded":"2024-09-13T05:00:44.130360+0000","last_fullsized":"2024-09-13T05:00:44.130360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"141'339","reported_seq":2686,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143833+0000","last_change":"2024-09-13T05:00:23.995154+0000","last_active":"2024-09-13T05:00:44.143833+0000","last_peered":"2024-09-13T05:00:44.143833+0000","last_clean":"2024-09-13T05:00:44.143833+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:44.143833+0000","last_undegraded":"2024-09-13T05:00:44.143833+0000","last_fullsized":"2024-09-13T05:00:44.143833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.131058+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:44.131058+0000","last_peered":"2024-09-13T05:00:44.131058+0000","last_clean":"2024-09-13T05:00:44.131058+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:44.131058+0000","last_undegraded":"2024-09-13T05:00:44.131058+0000","last_fullsized":"2024-09-13T05:00:44.131058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2240,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130942+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:44.130942+0000","last_peered":"2024-09-13T05:00:44.130942+0000","last_clean":"2024-09-13T05:00:44.130942+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:44.130942+0000","last_undegraded":"2024-09-13T05:00:44.130942+0000","last_fullsized":"2024-09-13T05:00:44.130942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130598+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:44.130598+0000","last_peered":"2024-09-13T05:00:44.130598+0000","last_clean":"2024-09-13T05:00:44.130598+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:44.130598+0000","last_undegraded":"2024-09-13T05:00:44.130598+0000","last_fullsized":"2024-09-13T05:00:44.130598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'633","reported_seq":1771,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143707+0000","last_change":"2024-09-13T05:00:42.129609+0000","last_active":"2024-09-13T05:00:44.143707+0000","last_peered":"2024-09-13T05:00:44.143707+0000","last_clean":"2024-09-13T05:00:44.143707+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:00:44.143707+0000","last_undegraded":"2024-09-13T05:00:44.143707+0000","last_fullsized":"2024-09-13T05:00:44.143707+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:35.254347+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132888+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:44.132888+0000","last_peered":"2024-09-13T05:00:44.132888+0000","last_clean":"2024-09-13T05:00:44.132888+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:44.132888+0000","last_undegraded":"2024-09-13T05:00:44.132888+0000","last_fullsized":"2024-09-13T05:00:44.132888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"135'558","reported_seq":2451,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577992+0000","last_change":"2024-09-13T05:00:23.939264+0000","last_active":"2024-09-13T05:00:44.577992+0000","last_peered":"2024-09-13T05:00:44.577992+0000","last_clean":"2024-09-13T05:00:44.577992+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:44.577992+0000","last_undegraded":"2024-09-13T05:00:44.577992+0000","last_fullsized":"2024-09-13T05:00:44.577992+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"141'925","reported_seq":4951,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578241+0000","last_change":"2024-09-13T05:00:23.954956+0000","last_active":"2024-09-13T05:00:44.578241+0000","last_peered":"2024-09-13T05:00:44.578241+0000","last_clean":"2024-09-13T05:00:44.578241+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:44.578241+0000","last_undegraded":"2024-09-13T05:00:44.578241+0000","last_fullsized":"2024-09-13T05:00:44.578241+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":156,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130483+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:44.130483+0000","last_peered":"2024-09-13T05:00:44.130483+0000","last_clean":"2024-09-13T05:00:44.130483+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:44.130483+0000","last_undegraded":"2024-09-13T05:00:44.130483+0000","last_fullsized":"2024-09-13T05:00:44.130483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"136'1217","reported_seq":2897,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132642+0000","last_change":"2024-09-13T05:00:23.937805+0000","last_active":"2024-09-13T05:00:44.132642+0000","last_peered":"2024-09-13T05:00:44.132642+0000","last_clean":"2024-09-13T05:00:44.132642+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:44.132642+0000","last_undegraded":"2024-09-13T05:00:44.132642+0000","last_fullsized":"2024-09-13T05:00:44.132642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.153709+0000","last_change":"2024-09-13T05:00:45.153709+0000","last_active":"2024-09-13T05:00:45.153709+0000","last_peered":"2024-09-13T05:00:45.153709+0000","last_clean":"2024-09-13T05:00:45.153709+0000","last_became_active":"2024-09-13T05:00:45.152844+0000","last_became_peered":"2024-09-13T05:00:45.152844+0000","last_unstale":"2024-09-13T05:00:45.153709+0000","last_undegraded":"2024-09-13T05:00:45.153709+0000","last_fullsized":"2024-09-13T05:00:45.153709+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:00.253343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"141'454","reported_seq":5162,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143352+0000","last_change":"2024-09-13T05:00:23.993994+0000","last_active":"2024-09-13T05:00:44.143352+0000","last_peered":"2024-09-13T05:00:44.143352+0000","last_clean":"2024-09-13T05:00:44.143352+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:44.143352+0000","last_undegraded":"2024-09-13T05:00:44.143352+0000","last_fullsized":"2024-09-13T05:00:44.143352+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129930+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:44.129930+0000","last_peered":"2024-09-13T05:00:44.129930+0000","last_clean":"2024-09-13T05:00:44.129930+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:44.129930+0000","last_undegraded":"2024-09-13T05:00:44.129930+0000","last_fullsized":"2024-09-13T05:00:44.129930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"140'1034","reported_seq":3339,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130908+0000","last_change":"2024-09-13T05:00:23.945801+0000","last_active":"2024-09-13T05:00:44.130908+0000","last_peered":"2024-09-13T05:00:44.130908+0000","last_clean":"2024-09-13T05:00:44.130908+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:44.130908+0000","last_undegraded":"2024-09-13T05:00:44.130908+0000","last_fullsized":"2024-09-13T05:00:44.130908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":155,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130159+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:44.130159+0000","last_peered":"2024-09-13T05:00:44.130159+0000","last_clean":"2024-09-13T05:00:44.130159+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:44.130159+0000","last_undegraded":"2024-09-13T05:00:44.130159+0000","last_fullsized":"2024-09-13T05:00:44.130159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":199,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579202+0000","last_change":"2024-09-13T05:00:19.959464+0000","last_active":"2024-09-13T05:00:44.579202+0000","last_peered":"2024-09-13T05:00:44.579202+0000","last_clean":"2024-09-13T05:00:44.579202+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:44.579202+0000","last_undegraded":"2024-09-13T05:00:44.579202+0000","last_fullsized":"2024-09-13T05:00:44.579202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:02.673583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"141'520","reported_seq":3954,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130538+0000","last_change":"2024-09-13T05:00:23.945803+0000","last_active":"2024-09-13T05:00:44.130538+0000","last_peered":"2024-09-13T05:00:44.130538+0000","last_clean":"2024-09-13T05:00:44.130538+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:44.130538+0000","last_undegraded":"2024-09-13T05:00:44.130538+0000","last_fullsized":"2024-09-13T05:00:44.130538+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"140'627","reported_seq":6115,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129970+0000","last_change":"2024-09-13T05:00:24.075211+0000","last_active":"2024-09-13T05:00:44.129970+0000","last_peered":"2024-09-13T05:00:44.129970+0000","last_clean":"2024-09-13T05:00:44.129970+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:44.129970+0000","last_undegraded":"2024-09-13T05:00:44.129970+0000","last_fullsized":"2024-09-13T05:00:44.129970+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"140'495","reported_seq":7322,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130370+0000","last_change":"2024-09-13T05:00:24.077894+0000","last_active":"2024-09-13T05:00:44.130370+0000","last_peered":"2024-09-13T05:00:44.130370+0000","last_clean":"2024-09-13T05:00:44.130370+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:00:44.130370+0000","last_undegraded":"2024-09-13T05:00:44.130370+0000","last_fullsized":"2024-09-13T05:00:44.130370+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:28:55.950141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"133'330","reported_seq":2155,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.159707+0000","last_change":"2024-09-13T05:00:45.159707+0000","last_active":"2024-09-13T05:00:45.159707+0000","last_peered":"2024-09-13T05:00:45.159707+0000","last_clean":"2024-09-13T05:00:45.159707+0000","last_became_active":"2024-09-13T05:00:45.157021+0000","last_became_peered":"2024-09-13T05:00:45.157021+0000","last_unstale":"2024-09-13T05:00:45.159707+0000","last_undegraded":"2024-09-13T05:00:45.159707+0000","last_fullsized":"2024-09-13T05:00:45.159707+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:06.713133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"141'698","reported_seq":1940,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578444+0000","last_change":"2024-09-13T05:00:23.957946+0000","last_active":"2024-09-13T05:00:44.578444+0000","last_peered":"2024-09-13T05:00:44.578444+0000","last_clean":"2024-09-13T05:00:44.578444+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:44.578444+0000","last_undegraded":"2024-09-13T05:00:44.578444+0000","last_fullsized":"2024-09-13T05:00:44.578444+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"141'472","reported_seq":5031,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130630+0000","last_change":"2024-09-13T05:00:24.078283+0000","last_active":"2024-09-13T05:00:44.130630+0000","last_peered":"2024-09-13T05:00:44.130630+0000","last_clean":"2024-09-13T05:00:44.130630+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:00:44.130630+0000","last_undegraded":"2024-09-13T05:00:44.130630+0000","last_fullsized":"2024-09-13T05:00:44.130630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:11.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'451","reported_seq":2074,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577945+0000","last_change":"2024-09-13T05:00:23.939918+0000","last_active":"2024-09-13T05:00:44.577945+0000","last_peered":"2024-09-13T05:00:44.577945+0000","last_clean":"2024-09-13T05:00:44.577945+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:44.577945+0000","last_undegraded":"2024-09-13T05:00:44.577945+0000","last_fullsized":"2024-09-13T05:00:44.577945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'855","reported_seq":8809,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578593+0000","last_change":"2024-09-13T05:00:23.955180+0000","last_active":"2024-09-13T05:00:44.578593+0000","last_peered":"2024-09-13T05:00:44.578593+0000","last_clean":"2024-09-13T05:00:44.578593+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:00:44.578593+0000","last_undegraded":"2024-09-13T05:00:44.578593+0000","last_fullsized":"2024-09-13T05:00:44.578593+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:05.706484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"137'449","reported_seq":7170,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579189+0000","last_change":"2024-09-13T05:00:23.938922+0000","last_active":"2024-09-13T05:00:44.579189+0000","last_peered":"2024-09-13T05:00:44.579189+0000","last_clean":"2024-09-13T05:00:44.579189+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:44.579189+0000","last_undegraded":"2024-09-13T05:00:44.579189+0000","last_fullsized":"2024-09-13T05:00:44.579189+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:32:00.161910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"140'410","reported_seq":5182,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129937+0000","last_change":"2024-09-13T05:00:24.075984+0000","last_active":"2024-09-13T05:00:44.129937+0000","last_peered":"2024-09-13T05:00:44.129937+0000","last_clean":"2024-09-13T05:00:44.129937+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:44.129937+0000","last_undegraded":"2024-09-13T05:00:44.129937+0000","last_fullsized":"2024-09-13T05:00:44.129937+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'456","reported_seq":3704,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143364+0000","last_change":"2024-09-13T05:00:23.995986+0000","last_active":"2024-09-13T05:00:44.143364+0000","last_peered":"2024-09-13T05:00:44.143364+0000","last_clean":"2024-09-13T05:00:44.143364+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:44.143364+0000","last_undegraded":"2024-09-13T05:00:44.143364+0000","last_fullsized":"2024-09-13T05:00:44.143364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"141'630","reported_seq":2062,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130029+0000","last_change":"2024-09-13T05:00:24.077226+0000","last_active":"2024-09-13T05:00:44.130029+0000","last_peered":"2024-09-13T05:00:44.130029+0000","last_clean":"2024-09-13T05:00:44.130029+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:44.130029+0000","last_undegraded":"2024-09-13T05:00:44.130029+0000","last_fullsized":"2024-09-13T05:00:44.130029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45280,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326560,"statfs":{"total":96636764160,"available":96590397440,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.89000000000000001}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.099},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":1.016}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":1.212},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.282},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.143}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.71199999999999997,"15min":0.72199999999999998},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1619999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.524}]},{"osd":1,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.746,"15min":0.754},"min":{"1min":0.44,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.202},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.337,"15min":1.337},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.94899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0409999999999999,"5min":5.343,"15min":5.343},"last":1.161},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.1519999999999999,"5min":1.278,"15min":1.278},"last":1.24}]},{"osd":3,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.88,"15min":0.89200000000000002},"min":{"1min":0.61599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.202,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.91700000000000004,"15min":0.93700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1599999999999999,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.407}]},{"osd":4,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93999999999999995,"15min":0.95399999999999996},"min":{"1min":0.53000000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.232,"5min":1.323,"15min":1.323},"last":1.02},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3520000000000001,"5min":1.575,"15min":1.575},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.93100000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6779999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.095},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.502,"15min":1.502},"last":1.365}]},{"osd":7,"last update":"Fri Sep 13 04:59:48 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.032,"15min":1.034},"min":{"1min":0.50600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.605},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97399999999999998,"15min":0.97299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":1.4399999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":3.972},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":0.95699999999999996}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":3.9009999999999998}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.083},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.0009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55136,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316704,"statfs":{"total":96636764160,"available":96580304896,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.83599999999999997}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.876}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.0529999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.2430000000000001}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.206}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.306},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2769999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59228,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312612,"statfs":{"total":96636764160,"available":96576114688,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.33600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.93700000000000006,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.32000000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.2829999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.60099999999999998,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.223,"15min":0.223},"max":{"1min":1.8939999999999999,"5min":3.113,"15min":3.113},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.92600000000000005,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.747}]},{"osd":3,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.624,"15min":0.58999999999999997},"min":{"1min":0.442,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.70799999999999996,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9199999999999999,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.87,"15min":0.83499999999999996},"min":{"1min":0.69999999999999996,"5min":0.374,"15min":0.374},"max":{"1min":2.0430000000000001,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.93000000000000005,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3560000000000001,"5min":1.861,"15min":1.861},"last":0.84899999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.71499999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4119999999999999,"5min":1.627,"15min":1.627},"last":1.008},{"interface":"front","average":{"1min":1.05,"5min":0.94799999999999995,"15min":0.92700000000000005},"min":{"1min":0.64600000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.9430000000000001,"5min":3.327,"15min":3.327},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.99199999999999999,"15min":1.0109999999999999},"min":{"1min":0.78500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.411,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.046,"5min":0.96499999999999997,"15min":0.96099999999999997},"min":{"1min":0.65000000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.405,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.97599999999999998}]},{"osd":7,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.006,"15min":1.02},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3819999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.048},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.66800000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4650000000000001,"5min":2.254,"15min":2.254},"last":1.1459999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59276,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312564,"statfs":{"total":96636764160,"available":96576065536,"internally_reserved":0,"allocated":1961984,"data_stored":1501653,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.90400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.317},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.075,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.79400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.34699999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.35099999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.503},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.81599999999999995,"15min":0.82499999999999996},"min":{"1min":0.53600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.131,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.91100000000000003,"15min":0.92700000000000005},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.1220000000000001,"5min":1.655,"15min":1.655},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.54300000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.60799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.1910000000000001,"5min":1.577,"15min":1.577},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.94599999999999995,"15min":0.95599999999999996},"min":{"1min":0.59499999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.738,"15min":1.738},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Sep 13 04:59:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1000000000000001,"5min":1.766,"15min":1.766},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3740000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.93200000000000005}]}]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":1.0760000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":1.024}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":1.0489999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":1.248},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.143}]}]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51096,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320744,"statfs":{"total":96636764160,"available":96584441856,"internally_reserved":0,"allocated":1978368,"data_stored":1501653,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.90900000000000003},"min":{"1min":0.48899999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2649999999999999,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.82699999999999996,"15min":0.80600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.77800000000000002,"15min":0.74099999999999999},"min":{"1min":0.46500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.177},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.75700000000000001,"15min":0.73499999999999999},"min":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.876}]},{"osd":2,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.48,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.05},{"interface":"front","average":{"1min":0.871,"5min":0.94999999999999996,"15min":0.996},"min":{"1min":0.65600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.107,"5min":4.633,"15min":4.633},"last":0.94399999999999995}]},{"osd":3,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.89300000000000002,"15min":0.85599999999999998},"min":{"1min":0.64300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.876,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.135,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.96199999999999997}]},{"osd":4,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89700000000000002,"15min":0.93500000000000005},"min":{"1min":0.45300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.204,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.88600000000000001,"15min":0.85999999999999999},"min":{"1min":0.60499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.3089999999999999,"5min":1.409,"15min":1.409},"last":1.1359999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4159999999999999,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.099,"15min":1.113},"min":{"1min":0.66300000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.601,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.089}]},{"osd":7,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.97499999999999998,"15min":0.94799999999999995},"min":{"1min":0.53200000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3859999999999999,"5min":1.454,"15min":1.454},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95099999999999996,"15min":0.91300000000000003},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.554,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.69899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:53.200 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:00:53.490 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:53.490 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":160,"fsid":"8850da77-4384-4335-9559-af2e805c4f72","created":"2024-09-13T04:57:34.164121+0000","modified":"2024-09-13T05:00:45.135423+0000","last_up_change":"2024-09-13T04:57:47.098606+0000","last_in_change":"2024-09-13T05:00:16.093048+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T04:57:48.984599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T04:57:52.756781+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T04:58:02.446076+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":80,"snap_epoch":141,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2fda29ca-3c6d-47ae-9244-8ccdf55f89f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6820","nonce":2346968854}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6824","nonce":2346968854}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6831","nonce":2346968854}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2346968854},{"type":"v1","addr":"172.21.15.1:6828","nonce":2346968854}]},"public_addr":"172.21.15.1:6820/2346968854","cluster_addr":"172.21.15.1:6824/2346968854","heartbeat_back_addr":"172.21.15.1:6831/2346968854","heartbeat_front_addr":"172.21.15.1:6828/2346968854","state":["exists","up"]},{"osd":1,"uuid":"9ca6e867-faf0-4295-b8eb-4c738dd26cb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6817","nonce":466532899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6819","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6821","nonce":466532899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6829","nonce":466532899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6823","nonce":466532899},{"type":"v1","addr":"172.21.15.1:6825","nonce":466532899}]},"public_addr":"172.21.15.1:6817/466532899","cluster_addr":"172.21.15.1:6821/466532899","heartbeat_back_addr":"172.21.15.1:6829/466532899","heartbeat_front_addr":"172.21.15.1:6825/466532899","state":["exists","up"]},{"osd":2,"uuid":"d8159400-8662-4567-8714-b48c2d8f3ad2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6801","nonce":343712578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6803","nonce":343712578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6807","nonce":343712578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":343712578},{"type":"v1","addr":"172.21.15.1:6805","nonce":343712578}]},"public_addr":"172.21.15.1:6801/343712578","cluster_addr":"172.21.15.1:6803/343712578","heartbeat_back_addr":"172.21.15.1:6807/343712578","heartbeat_front_addr":"172.21.15.1:6805/343712578","state":["exists","up"]},{"osd":3,"uuid":"13a3ea70-fa37-41d8-bae7-0203952b02cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6809","nonce":2021827348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6811","nonce":2021827348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6815","nonce":2021827348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2021827348},{"type":"v1","addr":"172.21.15.1:6813","nonce":2021827348}]},"public_addr":"172.21.15.1:6809/2021827348","cluster_addr":"172.21.15.1:6811/2021827348","heartbeat_back_addr":"172.21.15.1:6815/2021827348","heartbeat_front_addr":"172.21.15.1:6813/2021827348","state":["exists","up"]},{"osd":4,"uuid":"e896e6b5-c54a-405f-ab69-2a5b7b7a153e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6801","nonce":2183053562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6803","nonce":2183053562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6807","nonce":2183053562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2183053562},{"type":"v1","addr":"172.21.15.144:6805","nonce":2183053562}]},"public_addr":"172.21.15.144:6801/2183053562","cluster_addr":"172.21.15.144:6803/2183053562","heartbeat_back_addr":"172.21.15.144:6807/2183053562","heartbeat_front_addr":"172.21.15.144:6805/2183053562","state":["exists","up"]},{"osd":5,"uuid":"7de23072-7eda-43d6-9b92-d608ef82c6fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6817","nonce":2765053006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6819","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6821","nonce":2765053006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6828","nonce":2765053006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2765053006},{"type":"v1","addr":"172.21.15.144:6824","nonce":2765053006}]},"public_addr":"172.21.15.144:6817/2765053006","cluster_addr":"172.21.15.144:6821/2765053006","heartbeat_back_addr":"172.21.15.144:6828/2765053006","heartbeat_front_addr":"172.21.15.144:6824/2765053006","state":["exists","up"]},{"osd":6,"uuid":"8719fad9-81a0-46be-a546-db83bcace33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6820","nonce":2721913793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6825","nonce":2721913793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6831","nonce":2721913793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6827","nonce":2721913793},{"type":"v1","addr":"172.21.15.144:6829","nonce":2721913793}]},"public_addr":"172.21.15.144:6820/2721913793","cluster_addr":"172.21.15.144:6825/2721913793","heartbeat_back_addr":"172.21.15.144:6831/2721913793","heartbeat_front_addr":"172.21.15.144:6829/2721913793","state":["exists","up"]},{"osd":7,"uuid":"8bf0a351-e58e-4393-a6ce-fe70acf362aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6809","nonce":4202513770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6811","nonce":4202513770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6815","nonce":4202513770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":4202513770},{"type":"v1","addr":"172.21.15.144:6813","nonce":4202513770}]},"public_addr":"172.21.15.144:6809/4202513770","cluster_addr":"172.21.15.144:6811/4202513770","heartbeat_back_addr":"172.21.15.144:6815/4202513770","heartbeat_front_addr":"172.21.15.144:6813/4202513770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.092681+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.414226+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.368701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.370707+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.980085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:41.874193+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.219724+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T04:57:42.114987+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.6","osds":[6,4]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:00:54.508 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T05:00:54.508 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:00:54.671 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:54.671 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:00:54.671 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:54.687 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T05:00:54.973 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T05:00:54.992 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T05:00:54.992 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:00:55.144 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:55.145 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:00:55.145 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:55.161 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T05:00:55.454 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T05:00:55.474 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T05:00:55.474 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:00:55.630 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:55.631 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:00:55.631 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:55.647 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T05:00:55.940 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T05:00:55.960 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T05:00:55.960 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:00:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:56.115 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:00:56.115 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:56.132 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T05:00:56.427 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T05:00:56.448 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T05:00:56.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:00:56.609 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:56.609 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:00:56.609 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:56.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T05:00:56.919 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T05:00:56.937 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T05:00:56.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:00:57.093 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:57.094 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:00:57.094 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:57.110 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T05:00:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T05:00:57.415 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T05:00:57.415 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:00:57.574 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:57.574 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:00:57.574 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:57.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T05:00:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T05:00:57.891 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T05:00:57.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:00:58.033 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-13T05:00:58.033 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:00:58.033 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-13T05:00:58.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T05:00:58.339 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T05:00:58.359 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:00:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:00:58.663 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:00:58.682 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-13T05:00:56.955807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18650,"ondisk_log_size":18650,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413288,"kb_used_data":15088,"kb_used_omap":12,"kb_used_meta":398067,"kb_avail":754561432,"statfs":{"total":773094113280,"available":772670906368,"internally_reserved":0,"allocated":15450112,"data_stored":8470940,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":407621177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.812737"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3927,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133058+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:44.133058+0000","last_peered":"2024-09-13T05:00:44.133058+0000","last_clean":"2024-09-13T05:00:44.133058+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:44.133058+0000","last_undegraded":"2024-09-13T05:00:44.133058+0000","last_fullsized":"2024-09-13T05:00:44.133058+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:43.347419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"133'384","reported_seq":2268,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132882+0000","last_change":"2024-09-13T05:00:23.938537+0000","last_active":"2024-09-13T05:00:44.132882+0000","last_peered":"2024-09-13T05:00:44.132882+0000","last_clean":"2024-09-13T05:00:44.132882+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:00:44.132882+0000","last_undegraded":"2024-09-13T05:00:44.132882+0000","last_fullsized":"2024-09-13T05:00:44.132882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:35:32.060625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3814,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130405+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:44.130405+0000","last_peered":"2024-09-13T05:00:44.130405+0000","last_clean":"2024-09-13T05:00:44.130405+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:44.130405+0000","last_undegraded":"2024-09-13T05:00:44.130405+0000","last_fullsized":"2024-09-13T05:00:44.130405+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:25:49.102186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"141'604","reported_seq":2264,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579388+0000","last_change":"2024-09-13T05:00:23.938966+0000","last_active":"2024-09-13T05:00:44.579388+0000","last_peered":"2024-09-13T05:00:44.579388+0000","last_clean":"2024-09-13T05:00:44.579388+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:44.579388+0000","last_undegraded":"2024-09-13T05:00:44.579388+0000","last_fullsized":"2024-09-13T05:00:44.579388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:29:19.948030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5237,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130640+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:44.130640+0000","last_peered":"2024-09-13T05:00:44.130640+0000","last_clean":"2024-09-13T05:00:44.130640+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:44.130640+0000","last_undegraded":"2024-09-13T05:00:44.130640+0000","last_fullsized":"2024-09-13T05:00:44.130640+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:26:53.915111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"140'504","reported_seq":3991,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579242+0000","last_change":"2024-09-13T05:00:23.940610+0000","last_active":"2024-09-13T05:00:44.579242+0000","last_peered":"2024-09-13T05:00:44.579242+0000","last_clean":"2024-09-13T05:00:44.579242+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:00:44.579242+0000","last_undegraded":"2024-09-13T05:00:44.579242+0000","last_fullsized":"2024-09-13T05:00:44.579242+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:27:22.647601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"137'502","reported_seq":19475,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130628+0000","last_change":"2024-09-13T05:00:23.946784+0000","last_active":"2024-09-13T05:00:44.130628+0000","last_peered":"2024-09-13T05:00:44.130628+0000","last_clean":"2024-09-13T05:00:44.130628+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:00:44.130628+0000","last_undegraded":"2024-09-13T05:00:44.130628+0000","last_fullsized":"2024-09-13T05:00:44.130628+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.698921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"140'482","reported_seq":3079,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132942+0000","last_change":"2024-09-13T05:00:23.938257+0000","last_active":"2024-09-13T05:00:44.132942+0000","last_peered":"2024-09-13T05:00:44.132942+0000","last_clean":"2024-09-13T05:00:44.132942+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:00:44.132942+0000","last_undegraded":"2024-09-13T05:00:44.132942+0000","last_fullsized":"2024-09-13T05:00:44.132942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:55.174599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"136'332","reported_seq":3777,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.133187+0000","last_change":"2024-09-13T05:00:23.939322+0000","last_active":"2024-09-13T05:00:44.133187+0000","last_peered":"2024-09-13T05:00:44.133187+0000","last_clean":"2024-09-13T05:00:44.133187+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:00:44.133187+0000","last_undegraded":"2024-09-13T05:00:44.133187+0000","last_fullsized":"2024-09-13T05:00:44.133187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:09:23.924459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"140'560","reported_seq":2632,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577852+0000","last_change":"2024-09-13T05:00:23.939561+0000","last_active":"2024-09-13T05:00:44.577852+0000","last_peered":"2024-09-13T05:00:44.577852+0000","last_clean":"2024-09-13T05:00:44.577852+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:00:44.577852+0000","last_undegraded":"2024-09-13T05:00:44.577852+0000","last_fullsized":"2024-09-13T05:00:44.577852+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:40.109267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"133'431","reported_seq":1281,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578590+0000","last_change":"2024-09-13T05:00:23.957788+0000","last_active":"2024-09-13T05:00:44.578590+0000","last_peered":"2024-09-13T05:00:44.578590+0000","last_clean":"2024-09-13T05:00:44.578590+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:00:44.578590+0000","last_undegraded":"2024-09-13T05:00:44.578590+0000","last_fullsized":"2024-09-13T05:00:44.578590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:06:05.214184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"136'744","reported_seq":4417,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130067+0000","last_change":"2024-09-13T05:00:24.076712+0000","last_active":"2024-09-13T05:00:44.130067+0000","last_peered":"2024-09-13T05:00:44.130067+0000","last_clean":"2024-09-13T05:00:44.130067+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:00:44.130067+0000","last_undegraded":"2024-09-13T05:00:44.130067+0000","last_fullsized":"2024-09-13T05:00:44.130067+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:41:34.364056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130360+0000","last_change":"2024-09-13T04:57:56.155829+0000","last_active":"2024-09-13T05:00:44.130360+0000","last_peered":"2024-09-13T05:00:44.130360+0000","last_clean":"2024-09-13T05:00:44.130360+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:00:44.130360+0000","last_undegraded":"2024-09-13T05:00:44.130360+0000","last_fullsized":"2024-09-13T05:00:44.130360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:22:59.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"141'339","reported_seq":2686,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143833+0000","last_change":"2024-09-13T05:00:23.995154+0000","last_active":"2024-09-13T05:00:44.143833+0000","last_peered":"2024-09-13T05:00:44.143833+0000","last_clean":"2024-09-13T05:00:44.143833+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:00:44.143833+0000","last_undegraded":"2024-09-13T05:00:44.143833+0000","last_fullsized":"2024-09-13T05:00:44.143833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:39:22.382987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.131058+0000","last_change":"2024-09-13T04:57:56.154929+0000","last_active":"2024-09-13T05:00:44.131058+0000","last_peered":"2024-09-13T05:00:44.131058+0000","last_clean":"2024-09-13T05:00:44.131058+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:44.131058+0000","last_undegraded":"2024-09-13T05:00:44.131058+0000","last_fullsized":"2024-09-13T05:00:44.131058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:17.445360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2240,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130942+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:44.130942+0000","last_peered":"2024-09-13T05:00:44.130942+0000","last_clean":"2024-09-13T05:00:44.130942+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:44.130942+0000","last_undegraded":"2024-09-13T05:00:44.130942+0000","last_fullsized":"2024-09-13T05:00:44.130942+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:08:12.555256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130598+0000","last_change":"2024-09-13T04:57:56.155102+0000","last_active":"2024-09-13T05:00:44.130598+0000","last_peered":"2024-09-13T05:00:44.130598+0000","last_clean":"2024-09-13T05:00:44.130598+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:00:44.130598+0000","last_undegraded":"2024-09-13T05:00:44.130598+0000","last_fullsized":"2024-09-13T05:00:44.130598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:38:26.849352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"133'633","reported_seq":1771,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143707+0000","last_change":"2024-09-13T05:00:42.129609+0000","last_active":"2024-09-13T05:00:44.143707+0000","last_peered":"2024-09-13T05:00:44.143707+0000","last_clean":"2024-09-13T05:00:44.143707+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:00:44.143707+0000","last_undegraded":"2024-09-13T05:00:44.143707+0000","last_fullsized":"2024-09-13T05:00:44.143707+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:57:35.254347+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132888+0000","last_change":"2024-09-13T04:57:56.499872+0000","last_active":"2024-09-13T05:00:44.132888+0000","last_peered":"2024-09-13T05:00:44.132888+0000","last_clean":"2024-09-13T05:00:44.132888+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:44.132888+0000","last_undegraded":"2024-09-13T05:00:44.132888+0000","last_fullsized":"2024-09-13T05:00:44.132888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:05.637876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"135'558","reported_seq":2451,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577992+0000","last_change":"2024-09-13T05:00:23.939264+0000","last_active":"2024-09-13T05:00:44.577992+0000","last_peered":"2024-09-13T05:00:44.577992+0000","last_clean":"2024-09-13T05:00:44.577992+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:00:44.577992+0000","last_undegraded":"2024-09-13T05:00:44.577992+0000","last_fullsized":"2024-09-13T05:00:44.577992+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:48.727306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"141'925","reported_seq":4951,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578241+0000","last_change":"2024-09-13T05:00:23.954956+0000","last_active":"2024-09-13T05:00:44.578241+0000","last_peered":"2024-09-13T05:00:44.578241+0000","last_clean":"2024-09-13T05:00:44.578241+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:44.578241+0000","last_undegraded":"2024-09-13T05:00:44.578241+0000","last_fullsized":"2024-09-13T05:00:44.578241+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":925,"log_dups_size":0,"ondisk_log_size":925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:54:58.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":156,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130483+0000","last_change":"2024-09-13T04:57:56.154514+0000","last_active":"2024-09-13T05:00:44.130483+0000","last_peered":"2024-09-13T05:00:44.130483+0000","last_clean":"2024-09-13T05:00:44.130483+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:44.130483+0000","last_undegraded":"2024-09-13T05:00:44.130483+0000","last_fullsized":"2024-09-13T05:00:44.130483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:44.081610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"136'1217","reported_seq":2897,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.132642+0000","last_change":"2024-09-13T05:00:23.937805+0000","last_active":"2024-09-13T05:00:44.132642+0000","last_peered":"2024-09-13T05:00:44.132642+0000","last_clean":"2024-09-13T05:00:44.132642+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:00:44.132642+0000","last_undegraded":"2024-09-13T05:00:44.132642+0000","last_fullsized":"2024-09-13T05:00:44.132642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:39.511994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.153709+0000","last_change":"2024-09-13T05:00:45.153709+0000","last_active":"2024-09-13T05:00:45.153709+0000","last_peered":"2024-09-13T05:00:45.153709+0000","last_clean":"2024-09-13T05:00:45.153709+0000","last_became_active":"2024-09-13T05:00:45.152844+0000","last_became_peered":"2024-09-13T05:00:45.152844+0000","last_unstale":"2024-09-13T05:00:45.153709+0000","last_undegraded":"2024-09-13T05:00:45.153709+0000","last_fullsized":"2024-09-13T05:00:45.153709+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:15:00.253343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"141'454","reported_seq":5162,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143352+0000","last_change":"2024-09-13T05:00:23.993994+0000","last_active":"2024-09-13T05:00:44.143352+0000","last_peered":"2024-09-13T05:00:44.143352+0000","last_clean":"2024-09-13T05:00:44.143352+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:00:44.143352+0000","last_undegraded":"2024-09-13T05:00:44.143352+0000","last_fullsized":"2024-09-13T05:00:44.143352+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:28.882377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":154,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129930+0000","last_change":"2024-09-13T04:57:56.154502+0000","last_active":"2024-09-13T05:00:44.129930+0000","last_peered":"2024-09-13T05:00:44.129930+0000","last_clean":"2024-09-13T05:00:44.129930+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:00:44.129930+0000","last_undegraded":"2024-09-13T05:00:44.129930+0000","last_fullsized":"2024-09-13T05:00:44.129930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:58.677514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"140'1034","reported_seq":3339,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130908+0000","last_change":"2024-09-13T05:00:23.945801+0000","last_active":"2024-09-13T05:00:44.130908+0000","last_peered":"2024-09-13T05:00:44.130908+0000","last_clean":"2024-09-13T05:00:44.130908+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:00:44.130908+0000","last_undegraded":"2024-09-13T05:00:44.130908+0000","last_fullsized":"2024-09-13T05:00:44.130908+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:03:03.821381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":155,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130159+0000","last_change":"2024-09-13T04:57:56.155387+0000","last_active":"2024-09-13T05:00:44.130159+0000","last_peered":"2024-09-13T05:00:44.130159+0000","last_clean":"2024-09-13T05:00:44.130159+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:00:44.130159+0000","last_undegraded":"2024-09-13T05:00:44.130159+0000","last_fullsized":"2024-09-13T05:00:44.130159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:53.133684+0000","last_clean_scrub_stamp":"2024-09-13T04:57:53.133684+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:44:07.867403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":199,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579202+0000","last_change":"2024-09-13T05:00:19.959464+0000","last_active":"2024-09-13T05:00:44.579202+0000","last_peered":"2024-09-13T05:00:44.579202+0000","last_clean":"2024-09-13T05:00:44.579202+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:44.579202+0000","last_undegraded":"2024-09-13T05:00:44.579202+0000","last_fullsized":"2024-09-13T05:00:44.579202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:57:49.103658+0000","last_clean_scrub_stamp":"2024-09-13T04:57:49.103658+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:45:02.673583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"141'520","reported_seq":3954,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130538+0000","last_change":"2024-09-13T05:00:23.945803+0000","last_active":"2024-09-13T05:00:44.130538+0000","last_peered":"2024-09-13T05:00:44.130538+0000","last_clean":"2024-09-13T05:00:44.130538+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:00:44.130538+0000","last_undegraded":"2024-09-13T05:00:44.130538+0000","last_fullsized":"2024-09-13T05:00:44.130538+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:39.137535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"140'627","reported_seq":6115,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129970+0000","last_change":"2024-09-13T05:00:24.075211+0000","last_active":"2024-09-13T05:00:44.129970+0000","last_peered":"2024-09-13T05:00:44.129970+0000","last_clean":"2024-09-13T05:00:44.129970+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:00:44.129970+0000","last_undegraded":"2024-09-13T05:00:44.129970+0000","last_fullsized":"2024-09-13T05:00:44.129970+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:09:04.145464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"140'495","reported_seq":7322,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130370+0000","last_change":"2024-09-13T05:00:24.077894+0000","last_active":"2024-09-13T05:00:44.130370+0000","last_peered":"2024-09-13T05:00:44.130370+0000","last_clean":"2024-09-13T05:00:44.130370+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:00:44.130370+0000","last_undegraded":"2024-09-13T05:00:44.130370+0000","last_fullsized":"2024-09-13T05:00:44.130370+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:28:55.950141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"133'330","reported_seq":2155,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.159707+0000","last_change":"2024-09-13T05:00:45.159707+0000","last_active":"2024-09-13T05:00:45.159707+0000","last_peered":"2024-09-13T05:00:45.159707+0000","last_clean":"2024-09-13T05:00:45.159707+0000","last_became_active":"2024-09-13T05:00:45.157021+0000","last_became_peered":"2024-09-13T05:00:45.157021+0000","last_unstale":"2024-09-13T05:00:45.159707+0000","last_undegraded":"2024-09-13T05:00:45.159707+0000","last_fullsized":"2024-09-13T05:00:45.159707+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:14:06.713133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"141'698","reported_seq":1940,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578444+0000","last_change":"2024-09-13T05:00:23.957946+0000","last_active":"2024-09-13T05:00:44.578444+0000","last_peered":"2024-09-13T05:00:44.578444+0000","last_clean":"2024-09-13T05:00:44.578444+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:00:44.578444+0000","last_undegraded":"2024-09-13T05:00:44.578444+0000","last_fullsized":"2024-09-13T05:00:44.578444+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:39:40.991218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"141'472","reported_seq":5031,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130630+0000","last_change":"2024-09-13T05:00:24.078283+0000","last_active":"2024-09-13T05:00:44.130630+0000","last_peered":"2024-09-13T05:00:44.130630+0000","last_clean":"2024-09-13T05:00:44.130630+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:00:44.130630+0000","last_undegraded":"2024-09-13T05:00:44.130630+0000","last_fullsized":"2024-09-13T05:00:44.130630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:11.677613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'451","reported_seq":2074,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.577945+0000","last_change":"2024-09-13T05:00:23.939918+0000","last_active":"2024-09-13T05:00:44.577945+0000","last_peered":"2024-09-13T05:00:44.577945+0000","last_clean":"2024-09-13T05:00:44.577945+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:44.577945+0000","last_undegraded":"2024-09-13T05:00:44.577945+0000","last_fullsized":"2024-09-13T05:00:44.577945+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:36:30.672278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'855","reported_seq":8809,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.578593+0000","last_change":"2024-09-13T05:00:23.955180+0000","last_active":"2024-09-13T05:00:44.578593+0000","last_peered":"2024-09-13T05:00:44.578593+0000","last_clean":"2024-09-13T05:00:44.578593+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:00:44.578593+0000","last_undegraded":"2024-09-13T05:00:44.578593+0000","last_fullsized":"2024-09-13T05:00:44.578593+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:27:05.706484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"137'449","reported_seq":7170,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.579189+0000","last_change":"2024-09-13T05:00:23.938922+0000","last_active":"2024-09-13T05:00:44.579189+0000","last_peered":"2024-09-13T05:00:44.579189+0000","last_clean":"2024-09-13T05:00:44.579189+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:44.579189+0000","last_undegraded":"2024-09-13T05:00:44.579189+0000","last_fullsized":"2024-09-13T05:00:44.579189+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:32:00.161910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"140'410","reported_seq":5182,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.129937+0000","last_change":"2024-09-13T05:00:24.075984+0000","last_active":"2024-09-13T05:00:44.129937+0000","last_peered":"2024-09-13T05:00:44.129937+0000","last_clean":"2024-09-13T05:00:44.129937+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:00:44.129937+0000","last_undegraded":"2024-09-13T05:00:44.129937+0000","last_fullsized":"2024-09-13T05:00:44.129937+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:04:21.145383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"136'456","reported_seq":3704,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.143364+0000","last_change":"2024-09-13T05:00:23.995986+0000","last_active":"2024-09-13T05:00:44.143364+0000","last_peered":"2024-09-13T05:00:44.143364+0000","last_clean":"2024-09-13T05:00:44.143364+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:00:44.143364+0000","last_undegraded":"2024-09-13T05:00:44.143364+0000","last_fullsized":"2024-09-13T05:00:44.143364+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:19:11.458699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"141'630","reported_seq":2062,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-13T05:00:44.130029+0000","last_change":"2024-09-13T05:00:24.077226+0000","last_active":"2024-09-13T05:00:44.130029+0000","last_peered":"2024-09-13T05:00:44.130029+0000","last_clean":"2024-09-13T05:00:44.130029+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:00:44.130029+0000","last_undegraded":"2024-09-13T05:00:44.130029+0000","last_fullsized":"2024-09-13T05:00:44.130029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:40:40.304309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45280,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326560,"statfs":{"total":96636764160,"available":96590397440,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.51800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.85999999999999999,"15min":0.84099999999999997},"min":{"1min":0.61599999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":3.4169999999999998,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.83199999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.327,"15min":1.327},"last":0.91900000000000004}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.93799999999999994,"15min":0.96299999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1899999999999999,"5min":2.379,"15min":2.379},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.01,"5min":0.92400000000000004,"15min":0.91000000000000003},"min":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.5390000000000001,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2669999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.58299999999999996,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":0.997}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4710000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.056}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.89700000000000002,"15min":0.88100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":1.081},{"interface":"front","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.044},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3979999999999999,"5min":1.722,"15min":1.722},"last":1.147}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.012,"15min":1.0229999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.1379999999999999,"15min":1.163},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6259999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.226}]}]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.2010000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.091},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.42199999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.89700000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.755},"min":{"1min":0.38700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.202,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.44600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.218,"5min":1.337,"15min":1.337},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.91400000000000003,"15min":0.94799999999999995},"min":{"1min":0.43099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.8700000000000001,"5min":5.343,"15min":5.343},"last":1.044},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.48499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.0229999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.88400000000000001,"15min":0.89300000000000002},"min":{"1min":0.499,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92000000000000004,"15min":0.93799999999999994},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.90600000000000003,"15min":0.94299999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.103,"5min":1.323,"15min":1.323},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95399999999999996,"15min":0.97899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3220000000000001,"5min":1.575,"15min":1.575},"last":1.133}]},{"osd":5,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.93000000000000005,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6919999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.056,"5min":0.97499999999999998,"15min":0.97299999999999998},"min":{"1min":0.53400000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.0660000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0149999999999999,"15min":1.0289999999999999},"min":{"1min":0.435,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.605,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.03,"5min":0.98599999999999999,"15min":0.97699999999999998},"min":{"1min":0.59499999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5820000000000001,"5min":1.762,"15min":1.762},"last":0.92500000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.88100000000000001,"15min":0.91100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1459999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.94999999999999996}]},{"osd":1,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0860000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.89600000000000002,"15min":0.91500000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.159,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0609999999999999}]},{"osd":2,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.86699999999999999,"15min":0.88100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0860000000000001,"5min":1.607,"15min":1.607},"last":1.006},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0089999999999999,"5min":1.579,"15min":1.579},"last":1.2050000000000001}]},{"osd":3,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.3540000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1539999999999999,"5min":11.266,"15min":11.266},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.61399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.222,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.091}]},{"osd":5,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.306,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.873,"15min":0.88900000000000001},"min":{"1min":0.35299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.1830000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.006,"15min":1.022},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.274,"5min":1.752,"15min":1.752},"last":1.1739999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95499999999999996,"15min":0.96799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.026,"15min":1.046},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4059999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.98199999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55136,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316704,"statfs":{"total":96636764160,"available":96580304896,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.61399999999999999,"15min":0.63200000000000001},"min":{"1min":0.22700000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.26200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.032}]},{"osd":1,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.68300000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.079}]},{"osd":2,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.4379999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77600000000000002,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.6180000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.3979999999999999}]},{"osd":4,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.83399999999999996,"15min":0.85399999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.115},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.98899999999999999}]},{"osd":5,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.90200000000000002,"15min":0.91400000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.95999999999999996,"15min":0.99399999999999999},"min":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.054}]},{"osd":6,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98799999999999999,"15min":1.0029999999999999},"min":{"1min":0.48499999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.335},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0369999999999999,"15min":1.0649999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.879,"5min":1.994,"15min":1.994},"last":1.5609999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:53 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":1.0129999999999999,"15min":1.0449999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.004,"15min":1.018},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2549999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59228,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312612,"statfs":{"total":96636764160,"available":96576114688,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.33600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.93700000000000006,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.32000000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.2829999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.60099999999999998,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.223,"15min":0.223},"max":{"1min":1.8939999999999999,"5min":3.113,"15min":3.113},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.92600000000000005,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.747}]},{"osd":3,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.624,"15min":0.58999999999999997},"min":{"1min":0.442,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.70799999999999996,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9199999999999999,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.87,"15min":0.83499999999999996},"min":{"1min":0.69999999999999996,"5min":0.374,"15min":0.374},"max":{"1min":2.0430000000000001,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.93000000000000005,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3560000000000001,"5min":1.861,"15min":1.861},"last":0.84899999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.71499999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4119999999999999,"5min":1.627,"15min":1.627},"last":1.008},{"interface":"front","average":{"1min":1.05,"5min":0.94799999999999995,"15min":0.92700000000000005},"min":{"1min":0.64600000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.9430000000000001,"5min":3.327,"15min":3.327},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.99199999999999999,"15min":1.0109999999999999},"min":{"1min":0.78500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.411,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.046,"5min":0.96499999999999997,"15min":0.96099999999999997},"min":{"1min":0.65000000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.405,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.97599999999999998}]},{"osd":7,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.006,"15min":1.02},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3819999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.048},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.66800000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4650000000000001,"5min":2.254,"15min":2.254},"last":1.1459999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59276,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312564,"statfs":{"total":96636764160,"available":96576065536,"internally_reserved":0,"allocated":1961984,"data_stored":1501653,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.315,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.1259999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.39000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.96399999999999997,"5min":1.407,"15min":1.407},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.208,"5min":1.401,"15min":1.401},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.42599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1000000000000001,"5min":1.306,"15min":1.306},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.56399999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.2270000000000001,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.88800000000000001,"15min":0.91900000000000004},"min":{"1min":0.51300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0509999999999999,"5min":1.655,"15min":1.655},"last":0.76700000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85199999999999998,"15min":0.86299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.153,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.62,"5min":0.434,"15min":0.434},"max":{"1min":1.1850000000000001,"5min":1.49,"15min":1.49},"last":0.748}]},{"osd":6,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.88900000000000001,"15min":0.90800000000000003},"min":{"1min":0.62,"5min":0.496,"15min":0.496},"max":{"1min":1.2250000000000001,"5min":1.577,"15min":1.577},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.95299999999999996},"min":{"1min":0.65000000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3029999999999999,"5min":1.738,"15min":1.738},"last":0.872}]},{"osd":7,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.60499999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2290000000000001,"5min":1.766,"15min":1.766},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94799999999999995,"15min":0.95499999999999996},"min":{"1min":0.66600000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.349,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.85999999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.56999999999999995,"15min":0.58299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.83699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":1.365,"15min":1.5069999999999999},"min":{"1min":0.317,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.754,"5min":15.755000000000001,"15min":15.755000000000001},"last":1.0760000000000001}]},{"osd":2,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":1.427,"15min":1.5549999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.135,"5min":15.802,"15min":15.802},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.623,"15min":0.60699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.915,"5min":2.915,"15min":2.915},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.754,"15min":0.77100000000000002},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.165,"5min":1.26,"15min":1.26},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.591,"15min":1.726},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4430000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.874,"5min":0.78700000000000003,"15min":0.77300000000000002},"min":{"1min":0.52300000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.109,"5min":3.109,"15min":3.109},"last":1.024}]},{"osd":5,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.373,"15min":1.446},"min":{"1min":0.56000000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.1659999999999999,"5min":11.474,"15min":11.474},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.4610000000000001,"15min":1.5609999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1870000000000001,"5min":11.528,"15min":11.528},"last":1.0489999999999999}]},{"osd":6,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.94899999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.29,"5min":1.571,"15min":1.571},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.6839999999999999,"15min":1.8260000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2529999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":1.1719999999999999}]},{"osd":7,"last update":"Fri Sep 13 04:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.73,"15min":1.865},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3560000000000001,"5min":15.834,"15min":15.834},"last":1.248},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.93600000000000005,"15min":0.92300000000000004},"min":{"1min":0.65600000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.143}]}]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51096,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320744,"statfs":{"total":96636764160,"available":96584441856,"internally_reserved":0,"allocated":1978368,"data_stored":1501653,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.90900000000000003},"min":{"1min":0.48899999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2649999999999999,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.82699999999999996,"15min":0.80600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.77800000000000002,"15min":0.74099999999999999},"min":{"1min":0.46500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.177},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.75700000000000001,"15min":0.73499999999999999},"min":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.876}]},{"osd":2,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.48,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.05},{"interface":"front","average":{"1min":0.871,"5min":0.94999999999999996,"15min":0.996},"min":{"1min":0.65600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.107,"5min":4.633,"15min":4.633},"last":0.94399999999999995}]},{"osd":3,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.89300000000000002,"15min":0.85599999999999998},"min":{"1min":0.64300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.876,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.135,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.96199999999999997}]},{"osd":4,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89700000000000002,"15min":0.93500000000000005},"min":{"1min":0.45300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.204,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.88600000000000001,"15min":0.85999999999999999},"min":{"1min":0.60499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.3089999999999999,"5min":1.409,"15min":1.409},"last":1.1359999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4159999999999999,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.022,"5min":1.099,"15min":1.113},"min":{"1min":0.66300000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.601,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.089}]},{"osd":7,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.97499999999999998,"15min":0.94799999999999995},"min":{"1min":0.53200000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3859999999999999,"5min":1.454,"15min":1.454},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95099999999999996,"15min":0.91300000000000003},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.554,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.69899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:00:58.748 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.748 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.748 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.748 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.748 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.748 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.748 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T04:57:53.133684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T04:57:53.133684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.749 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T04:57:53.133684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T04:57:53.133684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T04:57:53.133684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T04:57:53.133684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.750 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.751 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T04:57:53.133684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.751 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.751 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T04:57:53.133684+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.751 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T04:57:49.103658+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=57, tm_sec=49, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.751 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.751 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.751 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.751 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:00:58.752 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T05:01:18.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:01:19.068 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:01:19.069 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:01:19.088 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-13T05:01:18.962403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19095,"ondisk_log_size":19095,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413232,"kb_used_data":15088,"kb_used_omap":12,"kb_used_meta":398067,"kb_avail":754561488,"statfs":{"total":773094113280,"available":772670963712,"internally_reserved":0,"allocated":15450112,"data_stored":8470940,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":407621177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003665"},"pg_stats":[{"pgid":"3.1f","version":"141'609","reported_seq":3928,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.147658+0000","last_change":"2024-09-13T05:00:23.938759+0000","last_active":"2024-09-13T05:00:45.147658+0000","last_peered":"2024-09-13T05:00:45.147658+0000","last_clean":"2024-09-13T05:00:45.147658+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:00:45.147658+0000","last_undegraded":"2024-09-13T05:00:45.147658+0000","last_fullsized":"2024-09-13T05:00:45.147658+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"160'398","reported_seq":2294,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:07.726244+0000","last_change":"2024-09-13T05:01:07.726497+0000","last_active":"2024-09-13T05:01:07.726244+0000","last_peered":"2024-09-13T05:01:07.726244+0000","last_clean":"2024-09-13T05:01:07.726244+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:01:07.726244+0000","last_undegraded":"2024-09-13T05:01:07.726244+0000","last_fullsized":"2024-09-13T05:01:07.726244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'398","last_scrub_stamp":"2024-09-13T05:01:07.726192+0000","last_deep_scrub":"160'398","last_deep_scrub_stamp":"2024-09-13T05:01:07.726192+0000","last_clean_scrub_stamp":"2024-09-13T05:01:07.726192+0000","objects_scrubbed":14,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:23.349006+0000","scrub_duration":0.020502889,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"141'535","reported_seq":3815,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.319238+0000","last_change":"2024-09-13T05:00:24.077198+0000","last_active":"2024-09-13T05:00:45.319238+0000","last_peered":"2024-09-13T05:00:45.319238+0000","last_clean":"2024-09-13T05:00:45.319238+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:00:45.319238+0000","last_undegraded":"2024-09-13T05:00:45.319238+0000","last_fullsized":"2024-09-13T05:00:45.319238+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"160'621","reported_seq":2293,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:59.599124+0000","last_change":"2024-09-13T05:00:59.599361+0000","last_active":"2024-09-13T05:00:59.599124+0000","last_peered":"2024-09-13T05:00:59.599124+0000","last_clean":"2024-09-13T05:00:59.599124+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:59.599124+0000","last_undegraded":"2024-09-13T05:00:59.599124+0000","last_fullsized":"2024-09-13T05:00:59.599124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'621","last_scrub_stamp":"2024-09-13T05:00:59.599071+0000","last_deep_scrub":"160'621","last_deep_scrub_stamp":"2024-09-13T05:00:59.599071+0000","last_clean_scrub_stamp":"2024-09-13T05:00:59.599071+0000","objects_scrubbed":17,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:48.833303+0000","scrub_duration":0.031728854000000001,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"136'544","reported_seq":5238,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.318144+0000","last_change":"2024-09-13T05:00:23.946380+0000","last_active":"2024-09-13T05:00:45.318144+0000","last_peered":"2024-09-13T05:00:45.318144+0000","last_clean":"2024-09-13T05:00:45.318144+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:00:45.318144+0000","last_undegraded":"2024-09-13T05:00:45.318144+0000","last_fullsized":"2024-09-13T05:00:45.318144+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"160'518","reported_seq":4017,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:03.585059+0000","last_change":"2024-09-13T05:01:03.585288+0000","last_active":"2024-09-13T05:01:03.585059+0000","last_peered":"2024-09-13T05:01:03.585059+0000","last_clean":"2024-09-13T05:01:03.585059+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:01:03.585059+0000","last_undegraded":"2024-09-13T05:01:03.585059+0000","last_fullsized":"2024-09-13T05:01:03.585059+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'518","last_scrub_stamp":"2024-09-13T05:01:03.585006+0000","last_deep_scrub":"160'518","last_deep_scrub_stamp":"2024-09-13T05:01:03.585006+0000","last_clean_scrub_stamp":"2024-09-13T05:01:03.585006+0000","objects_scrubbed":14,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:43.454127+0000","scrub_duration":0.026602105000000001,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"160'519","reported_seq":19504,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:16.240834+0000","last_change":"2024-09-13T05:01:16.241169+0000","last_active":"2024-09-13T05:01:16.240834+0000","last_peered":"2024-09-13T05:01:16.240834+0000","last_clean":"2024-09-13T05:01:16.240834+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:01:16.240834+0000","last_undegraded":"2024-09-13T05:01:16.240834+0000","last_fullsized":"2024-09-13T05:01:16.240834+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'519","last_scrub_stamp":"2024-09-13T05:01:16.240778+0000","last_deep_scrub":"160'519","last_deep_scrub_stamp":"2024-09-13T05:01:16.240778+0000","last_clean_scrub_stamp":"2024-09-13T05:01:16.240778+0000","objects_scrubbed":17,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:08:41.139301+0000","scrub_duration":0.031141914999999999,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"160'496","reported_seq":3105,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:05.754297+0000","last_change":"2024-09-13T05:01:05.754580+0000","last_active":"2024-09-13T05:01:05.754297+0000","last_peered":"2024-09-13T05:01:05.754297+0000","last_clean":"2024-09-13T05:01:05.754297+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:01:05.754297+0000","last_undegraded":"2024-09-13T05:01:05.754297+0000","last_fullsized":"2024-09-13T05:01:05.754297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'496","last_scrub_stamp":"2024-09-13T05:01:05.754245+0000","last_deep_scrub":"160'496","last_deep_scrub_stamp":"2024-09-13T05:01:05.754245+0000","last_clean_scrub_stamp":"2024-09-13T05:01:05.754245+0000","objects_scrubbed":14,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:53.100903+0000","scrub_duration":0.024650089,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"160'348","reported_seq":3805,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:11.702285+0000","last_change":"2024-09-13T05:01:11.702577+0000","last_active":"2024-09-13T05:01:11.702285+0000","last_peered":"2024-09-13T05:01:11.702285+0000","last_clean":"2024-09-13T05:01:11.702285+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:01:11.702285+0000","last_undegraded":"2024-09-13T05:01:11.702285+0000","last_fullsized":"2024-09-13T05:01:11.702285+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'348","last_scrub_stamp":"2024-09-13T05:01:11.702213+0000","last_deep_scrub":"160'348","last_deep_scrub_stamp":"2024-09-13T05:01:11.702213+0000","last_clean_scrub_stamp":"2024-09-13T05:01:11.702213+0000","objects_scrubbed":16,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:36.853328+0000","scrub_duration":0.02272708,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"160'570","reported_seq":2654,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:00.438428+0000","last_change":"2024-09-13T05:01:00.438707+0000","last_active":"2024-09-13T05:01:00.438428+0000","last_peered":"2024-09-13T05:01:00.438428+0000","last_clean":"2024-09-13T05:01:00.438428+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:01:00.438428+0000","last_undegraded":"2024-09-13T05:01:00.438428+0000","last_fullsized":"2024-09-13T05:01:00.438428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'570","last_scrub_stamp":"2024-09-13T05:01:00.438366+0000","last_deep_scrub":"160'570","last_deep_scrub_stamp":"2024-09-13T05:01:00.438366+0000","last_clean_scrub_stamp":"2024-09-13T05:01:00.438366+0000","objects_scrubbed":10,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:43:38.046169+0000","scrub_duration":0.0221271,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"160'440","reported_seq":1302,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:03.856773+0000","last_change":"2024-09-13T05:01:03.857080+0000","last_active":"2024-09-13T05:01:03.856773+0000","last_peered":"2024-09-13T05:01:03.856773+0000","last_clean":"2024-09-13T05:01:03.856773+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:01:03.856773+0000","last_undegraded":"2024-09-13T05:01:03.856773+0000","last_fullsized":"2024-09-13T05:01:03.856773+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'440","last_scrub_stamp":"2024-09-13T05:01:03.856711+0000","last_deep_scrub":"160'440","last_deep_scrub_stamp":"2024-09-13T05:01:03.856711+0000","last_clean_scrub_stamp":"2024-09-13T05:01:03.856711+0000","objects_scrubbed":9,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:14.107366+0000","scrub_duration":0.016563472999999999,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"160'760","reported_seq":4445,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:14.245661+0000","last_change":"2024-09-13T05:01:14.245987+0000","last_active":"2024-09-13T05:01:14.245661+0000","last_peered":"2024-09-13T05:01:14.245661+0000","last_clean":"2024-09-13T05:01:14.245661+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:01:14.245661+0000","last_undegraded":"2024-09-13T05:01:14.245661+0000","last_fullsized":"2024-09-13T05:01:14.245661+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'760","last_scrub_stamp":"2024-09-13T05:01:14.245607+0000","last_deep_scrub":"160'760","last_deep_scrub_stamp":"2024-09-13T05:01:14.245607+0000","last_clean_scrub_stamp":"2024-09-13T05:01:14.245607+0000","objects_scrubbed":16,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:53:23.500615+0000","scrub_duration":0.031691538999999998,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:02.240728+0000","last_change":"2024-09-13T05:01:02.240954+0000","last_active":"2024-09-13T05:01:02.240728+0000","last_peered":"2024-09-13T05:01:02.240728+0000","last_clean":"2024-09-13T05:01:02.240728+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:01:02.240728+0000","last_undegraded":"2024-09-13T05:01:02.240728+0000","last_fullsized":"2024-09-13T05:01:02.240728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:02.240675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:02.240675+0000","last_clean_scrub_stamp":"2024-09-13T05:01:02.240675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:57:03.447335+0000","scrub_duration":0.0029906199999999998,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"160'356","reported_seq":2715,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:06.563872+0000","last_change":"2024-09-13T05:01:06.564169+0000","last_active":"2024-09-13T05:01:06.563872+0000","last_peered":"2024-09-13T05:01:06.563872+0000","last_clean":"2024-09-13T05:01:06.563872+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:01:06.563872+0000","last_undegraded":"2024-09-13T05:01:06.563872+0000","last_fullsized":"2024-09-13T05:01:06.563872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'356","last_scrub_stamp":"2024-09-13T05:01:06.563812+0000","last_deep_scrub":"160'356","last_deep_scrub_stamp":"2024-09-13T05:01:06.563812+0000","last_clean_scrub_stamp":"2024-09-13T05:01:06.563812+0000","objects_scrubbed":17,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:19:27.190719+0000","scrub_duration":0.028154248,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:59.333515+0000","last_change":"2024-09-13T05:00:59.333741+0000","last_active":"2024-09-13T05:00:59.333515+0000","last_peered":"2024-09-13T05:00:59.333515+0000","last_clean":"2024-09-13T05:00:59.333515+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:59.333515+0000","last_undegraded":"2024-09-13T05:00:59.333515+0000","last_fullsized":"2024-09-13T05:00:59.333515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:00:59.333468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:00:59.333468+0000","last_clean_scrub_stamp":"2024-09-13T05:00:59.333468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:38:12.476332+0000","scrub_duration":0.0032761819999999999,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"135'746","reported_seq":2241,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:45.318254+0000","last_change":"2024-09-13T05:00:23.946447+0000","last_active":"2024-09-13T05:00:45.318254+0000","last_peered":"2024-09-13T05:00:45.318254+0000","last_clean":"2024-09-13T05:00:45.318254+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:00:45.318254+0000","last_undegraded":"2024-09-13T05:00:45.318254+0000","last_fullsized":"2024-09-13T05:00:45.318254+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T04:58:03.193961+0000","last_clean_scrub_stamp":"2024-09-13T04:58:03.193961+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:03.206685+0000","last_change":"2024-09-13T05:01:03.206910+0000","last_active":"2024-09-13T05:01:03.206685+0000","last_peered":"2024-09-13T05:01:03.206685+0000","last_clean":"2024-09-13T05:01:03.206685+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:01:03.206685+0000","last_undegraded":"2024-09-13T05:01:03.206685+0000","last_fullsized":"2024-09-13T05:01:03.206685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:03.206631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:03.206631+0000","last_clean_scrub_stamp":"2024-09-13T05:01:03.206631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:48:34.855796+0000","scrub_duration":0.0032981880000000001,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"160'658","reported_seq":1810,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:11.658844+0000","last_change":"2024-09-13T05:01:11.659058+0000","last_active":"2024-09-13T05:01:11.658844+0000","last_peered":"2024-09-13T05:01:11.658844+0000","last_clean":"2024-09-13T05:01:11.658844+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:01:11.658844+0000","last_undegraded":"2024-09-13T05:01:11.658844+0000","last_fullsized":"2024-09-13T05:01:11.658844+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'658","last_scrub_stamp":"2024-09-13T05:01:11.658787+0000","last_deep_scrub":"160'658","last_deep_scrub_stamp":"2024-09-13T05:01:11.658787+0000","last_clean_scrub_stamp":"2024-09-13T05:01:11.658787+0000","objects_scrubbed":25,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:37:52.619580+0000","scrub_duration":0.045409115,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.589479+0000","last_change":"2024-09-13T05:00:57.589690+0000","last_active":"2024-09-13T05:00:57.589479+0000","last_peered":"2024-09-13T05:00:57.589479+0000","last_clean":"2024-09-13T05:00:57.589479+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:57.589479+0000","last_undegraded":"2024-09-13T05:00:57.589479+0000","last_fullsized":"2024-09-13T05:00:57.589479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:00:57.589425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:00:57.589425+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.589425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:25:38.515636+0000","scrub_duration":0.002791155,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"160'575","reported_seq":2480,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:05.417896+0000","last_change":"2024-09-13T05:01:05.418152+0000","last_active":"2024-09-13T05:01:05.417896+0000","last_peered":"2024-09-13T05:01:05.417896+0000","last_clean":"2024-09-13T05:01:05.417896+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:01:05.417896+0000","last_undegraded":"2024-09-13T05:01:05.417896+0000","last_fullsized":"2024-09-13T05:01:05.417896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'575","last_scrub_stamp":"2024-09-13T05:01:05.417814+0000","last_deep_scrub":"160'575","last_deep_scrub_stamp":"2024-09-13T05:01:05.417814+0000","last_clean_scrub_stamp":"2024-09-13T05:01:05.417814+0000","objects_scrubbed":17,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:21.482343+0000","scrub_duration":0.027707463000000002,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'942","reported_seq":4980,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.009814+0000","last_change":"2024-09-13T05:00:57.009937+0000","last_active":"2024-09-13T05:00:57.009814+0000","last_peered":"2024-09-13T05:00:57.009814+0000","last_clean":"2024-09-13T05:00:57.009814+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:57.009814+0000","last_undegraded":"2024-09-13T05:00:57.009814+0000","last_fullsized":"2024-09-13T05:00:57.009814+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'942","last_scrub_stamp":"2024-09-13T05:00:57.009788+0000","last_deep_scrub":"160'942","last_deep_scrub_stamp":"2024-09-13T05:00:57.009788+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.009788+0000","objects_scrubbed":17,"log_size":942,"log_dups_size":0,"ondisk_log_size":942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:37:03.234330+0000","scrub_duration":0.015432392,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"160'3","reported_seq":169,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.246290+0000","last_change":"2024-09-13T05:00:57.246365+0000","last_active":"2024-09-13T05:00:57.246290+0000","last_peered":"2024-09-13T05:00:57.246290+0000","last_clean":"2024-09-13T05:00:57.246290+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:57.246290+0000","last_undegraded":"2024-09-13T05:00:57.246290+0000","last_fullsized":"2024-09-13T05:00:57.246290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'3","last_scrub_stamp":"2024-09-13T05:00:57.246270+0000","last_deep_scrub":"160'3","last_deep_scrub_stamp":"2024-09-13T05:00:57.246270+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.246270+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:09.860854+0000","scrub_duration":0.0041561840000000003,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"160'1238","reported_seq":2930,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:14.745766+0000","last_change":"2024-09-13T05:01:14.746036+0000","last_active":"2024-09-13T05:01:14.745766+0000","last_peered":"2024-09-13T05:01:14.745766+0000","last_clean":"2024-09-13T05:01:14.745766+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:01:14.745766+0000","last_undegraded":"2024-09-13T05:01:14.745766+0000","last_fullsized":"2024-09-13T05:01:14.745766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1238","last_scrub_stamp":"2024-09-13T05:01:14.745708+0000","last_deep_scrub":"160'1238","last_deep_scrub_stamp":"2024-09-13T05:01:14.745708+0000","last_clean_scrub_stamp":"2024-09-13T05:01:14.745708+0000","objects_scrubbed":21,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:16:16.291852+0000","scrub_duration":0.044443238000000003,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":153,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:59.172290+0000","last_change":"2024-09-13T05:00:59.172486+0000","last_active":"2024-09-13T05:00:59.172290+0000","last_peered":"2024-09-13T05:00:59.172290+0000","last_clean":"2024-09-13T05:00:59.172290+0000","last_became_active":"2024-09-13T05:00:45.152844+0000","last_became_peered":"2024-09-13T05:00:45.152844+0000","last_unstale":"2024-09-13T05:00:59.172290+0000","last_undegraded":"2024-09-13T05:00:59.172290+0000","last_fullsized":"2024-09-13T05:00:59.172290+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:00:59.172238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:00:59.172238+0000","last_clean_scrub_stamp":"2024-09-13T05:00:59.172238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:16:00.503765+0000","scrub_duration":0.002972592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"160'465","reported_seq":5185,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:03.569553+0000","last_change":"2024-09-13T05:01:03.569863+0000","last_active":"2024-09-13T05:01:03.569553+0000","last_peered":"2024-09-13T05:01:03.569553+0000","last_clean":"2024-09-13T05:01:03.569553+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:01:03.569553+0000","last_undegraded":"2024-09-13T05:01:03.569553+0000","last_fullsized":"2024-09-13T05:01:03.569553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'465","last_scrub_stamp":"2024-09-13T05:01:03.569497+0000","last_deep_scrub":"160'465","last_deep_scrub_stamp":"2024-09-13T05:01:03.569497+0000","last_clean_scrub_stamp":"2024-09-13T05:01:03.569497+0000","objects_scrubbed":11,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:50.070105+0000","scrub_duration":0.019109181999999999,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:05.202580+0000","last_change":"2024-09-13T05:01:05.202792+0000","last_active":"2024-09-13T05:01:05.202580+0000","last_peered":"2024-09-13T05:01:05.202580+0000","last_clean":"2024-09-13T05:01:05.202580+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:01:05.202580+0000","last_undegraded":"2024-09-13T05:01:05.202580+0000","last_fullsized":"2024-09-13T05:01:05.202580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:05.202530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:05.202530+0000","last_clean_scrub_stamp":"2024-09-13T05:01:05.202530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:33:06.154178+0000","scrub_duration":0.0035771919999999999,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"160'1058","reported_seq":3375,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:17.278641+0000","last_change":"2024-09-13T05:01:17.278877+0000","last_active":"2024-09-13T05:01:17.278641+0000","last_peered":"2024-09-13T05:01:17.278641+0000","last_clean":"2024-09-13T05:01:17.278641+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:01:17.278641+0000","last_undegraded":"2024-09-13T05:01:17.278641+0000","last_fullsized":"2024-09-13T05:01:17.278641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1058","last_scrub_stamp":"2024-09-13T05:01:17.278588+0000","last_deep_scrub":"160'1058","last_deep_scrub_stamp":"2024-09-13T05:01:17.278588+0000","last_clean_scrub_stamp":"2024-09-13T05:01:17.278588+0000","objects_scrubbed":24,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:02.679032+0000","scrub_duration":0.038963978000000003,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"160'2","reported_seq":168,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:06.202282+0000","last_change":"2024-09-13T05:01:06.202514+0000","last_active":"2024-09-13T05:01:06.202282+0000","last_peered":"2024-09-13T05:01:06.202282+0000","last_clean":"2024-09-13T05:01:06.202282+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:01:06.202282+0000","last_undegraded":"2024-09-13T05:01:06.202282+0000","last_fullsized":"2024-09-13T05:01:06.202282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'2","last_scrub_stamp":"2024-09-13T05:01:06.202195+0000","last_deep_scrub":"160'2","last_deep_scrub_stamp":"2024-09-13T05:01:06.202195+0000","last_clean_scrub_stamp":"2024-09-13T05:01:06.202195+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:13:10.718476+0000","scrub_duration":0.0061865330000000001,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"160'119","reported_seq":213,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:56.548750+0000","last_change":"2024-09-13T05:00:56.548894+0000","last_active":"2024-09-13T05:00:56.548750+0000","last_peered":"2024-09-13T05:00:56.548750+0000","last_clean":"2024-09-13T05:00:56.548750+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:56.548750+0000","last_undegraded":"2024-09-13T05:00:56.548750+0000","last_fullsized":"2024-09-13T05:00:56.548750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-09-13T05:00:56.548706+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-09-13T05:00:56.548706+0000","last_clean_scrub_stamp":"2024-09-13T05:00:56.548706+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:56:22.004941+0000","scrub_duration":0.011268804,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"160'537","reported_seq":3983,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:14.168194+0000","last_change":"2024-09-13T05:01:14.168472+0000","last_active":"2024-09-13T05:01:14.168194+0000","last_peered":"2024-09-13T05:01:14.168194+0000","last_clean":"2024-09-13T05:01:14.168194+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:01:14.168194+0000","last_undegraded":"2024-09-13T05:01:14.168194+0000","last_fullsized":"2024-09-13T05:01:14.168194+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'537","last_scrub_stamp":"2024-09-13T05:01:14.168137+0000","last_deep_scrub":"160'537","last_deep_scrub_stamp":"2024-09-13T05:01:14.168137+0000","last_clean_scrub_stamp":"2024-09-13T05:01:14.168137+0000","objects_scrubbed":17,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:48:49.345993+0000","scrub_duration":0.030468004,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"160'640","reported_seq":6140,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:13.190342+0000","last_change":"2024-09-13T05:01:13.190585+0000","last_active":"2024-09-13T05:01:13.190342+0000","last_peered":"2024-09-13T05:01:13.190342+0000","last_clean":"2024-09-13T05:01:13.190342+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:01:13.190342+0000","last_undegraded":"2024-09-13T05:01:13.190342+0000","last_fullsized":"2024-09-13T05:01:13.190342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'640","last_scrub_stamp":"2024-09-13T05:01:13.190281+0000","last_deep_scrub":"160'640","last_deep_scrub_stamp":"2024-09-13T05:01:13.190281+0000","last_clean_scrub_stamp":"2024-09-13T05:01:13.190281+0000","objects_scrubbed":13,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:58.704336+0000","scrub_duration":0.022449542999999999,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"160'506","reported_seq":7345,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:11.196208+0000","last_change":"2024-09-13T05:01:11.196470+0000","last_active":"2024-09-13T05:01:11.196208+0000","last_peered":"2024-09-13T05:01:11.196208+0000","last_clean":"2024-09-13T05:01:11.196208+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:01:11.196208+0000","last_undegraded":"2024-09-13T05:01:11.196208+0000","last_fullsized":"2024-09-13T05:01:11.196208+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'506","last_scrub_stamp":"2024-09-13T05:01:11.196152+0000","last_deep_scrub":"160'506","last_deep_scrub_stamp":"2024-09-13T05:01:11.196152+0000","last_clean_scrub_stamp":"2024-09-13T05:01:11.196152+0000","objects_scrubbed":11,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:47.070920+0000","scrub_duration":0.022952017000000002,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"160'343","reported_seq":2180,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:02.124222+0000","last_change":"2024-09-13T05:01:02.124450+0000","last_active":"2024-09-13T05:01:02.124222+0000","last_peered":"2024-09-13T05:01:02.124222+0000","last_clean":"2024-09-13T05:01:02.124222+0000","last_became_active":"2024-09-13T05:00:45.157021+0000","last_became_peered":"2024-09-13T05:00:45.157021+0000","last_unstale":"2024-09-13T05:01:02.124222+0000","last_undegraded":"2024-09-13T05:01:02.124222+0000","last_fullsized":"2024-09-13T05:01:02.124222+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'343","last_scrub_stamp":"2024-09-13T05:01:02.124166+0000","last_deep_scrub":"160'343","last_deep_scrub_stamp":"2024-09-13T05:01:02.124166+0000","last_clean_scrub_stamp":"2024-09-13T05:01:02.124166+0000","objects_scrubbed":13,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:01:01.825237+0000","scrub_duration":0.022449429999999999,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"160'715","reported_seq":1969,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:06.920739+0000","last_change":"2024-09-13T05:01:06.920974+0000","last_active":"2024-09-13T05:01:06.920739+0000","last_peered":"2024-09-13T05:01:06.920739+0000","last_clean":"2024-09-13T05:01:06.920739+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:01:06.920739+0000","last_undegraded":"2024-09-13T05:01:06.920739+0000","last_fullsized":"2024-09-13T05:01:06.920739+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'715","last_scrub_stamp":"2024-09-13T05:01:06.920687+0000","last_deep_scrub":"160'715","last_deep_scrub_stamp":"2024-09-13T05:01:06.920687+0000","last_clean_scrub_stamp":"2024-09-13T05:01:06.920687+0000","objects_scrubbed":17,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:32.500994+0000","scrub_duration":0.028190621999999999,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"160'483","reported_seq":5054,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:15.232289+0000","last_change":"2024-09-13T05:01:15.232527+0000","last_active":"2024-09-13T05:01:15.232289+0000","last_peered":"2024-09-13T05:01:15.232289+0000","last_clean":"2024-09-13T05:01:15.232289+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:01:15.232289+0000","last_undegraded":"2024-09-13T05:01:15.232289+0000","last_fullsized":"2024-09-13T05:01:15.232289+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'483","last_scrub_stamp":"2024-09-13T05:01:15.232236+0000","last_deep_scrub":"160'483","last_deep_scrub_stamp":"2024-09-13T05:01:15.232236+0000","last_clean_scrub_stamp":"2024-09-13T05:01:15.232236+0000","objects_scrubbed":11,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:08:04.637216+0000","scrub_duration":0.021577643000000001,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"160'468","reported_seq":2103,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.410580+0000","last_change":"2024-09-13T05:00:57.410687+0000","last_active":"2024-09-13T05:00:57.410580+0000","last_peered":"2024-09-13T05:00:57.410580+0000","last_clean":"2024-09-13T05:00:57.410580+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:57.410580+0000","last_undegraded":"2024-09-13T05:00:57.410580+0000","last_fullsized":"2024-09-13T05:00:57.410580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'468","last_scrub_stamp":"2024-09-13T05:00:57.410559+0000","last_deep_scrub":"160'468","last_deep_scrub_stamp":"2024-09-13T05:00:57.410559+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.410559+0000","objects_scrubbed":17,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:57:25.177746+0000","scrub_duration":0.014674352,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"160'870","reported_seq":8836,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:07.875995+0000","last_change":"2024-09-13T05:01:07.876258+0000","last_active":"2024-09-13T05:01:07.875995+0000","last_peered":"2024-09-13T05:01:07.875995+0000","last_clean":"2024-09-13T05:01:07.875995+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:01:07.875995+0000","last_undegraded":"2024-09-13T05:01:07.875995+0000","last_fullsized":"2024-09-13T05:01:07.875995+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'870","last_scrub_stamp":"2024-09-13T05:01:07.875938+0000","last_deep_scrub":"160'870","last_deep_scrub_stamp":"2024-09-13T05:01:07.875938+0000","last_clean_scrub_stamp":"2024-09-13T05:01:07.875938+0000","objects_scrubbed":15,"log_size":870,"log_dups_size":0,"ondisk_log_size":870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:45.718013+0000","scrub_duration":0.030610100000000001,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"160'471","reported_seq":7204,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.561850+0000","last_change":"2024-09-13T05:00:57.561976+0000","last_active":"2024-09-13T05:00:57.561850+0000","last_peered":"2024-09-13T05:00:57.561850+0000","last_clean":"2024-09-13T05:00:57.561850+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:57.561850+0000","last_undegraded":"2024-09-13T05:00:57.561850+0000","last_fullsized":"2024-09-13T05:00:57.561850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'471","last_scrub_stamp":"2024-09-13T05:00:57.561829+0000","last_deep_scrub":"160'471","last_deep_scrub_stamp":"2024-09-13T05:00:57.561829+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.561829+0000","objects_scrubbed":22,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:02.052421+0000","scrub_duration":0.024728844,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"160'428","reported_seq":5212,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:16.252436+0000","last_change":"2024-09-13T05:01:16.252761+0000","last_active":"2024-09-13T05:01:16.252436+0000","last_peered":"2024-09-13T05:01:16.252436+0000","last_clean":"2024-09-13T05:01:16.252436+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:01:16.252436+0000","last_undegraded":"2024-09-13T05:01:16.252436+0000","last_fullsized":"2024-09-13T05:01:16.252436+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'428","last_scrub_stamp":"2024-09-13T05:01:16.252377+0000","last_deep_scrub":"160'428","last_deep_scrub_stamp":"2024-09-13T05:01:16.252377+0000","last_clean_scrub_stamp":"2024-09-13T05:01:16.252377+0000","objects_scrubbed":18,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:15:27.453430+0000","scrub_duration":0.033938712000000003,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"160'470","reported_seq":3730,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:01.606515+0000","last_change":"2024-09-13T05:01:01.606773+0000","last_active":"2024-09-13T05:01:01.606515+0000","last_peered":"2024-09-13T05:01:01.606515+0000","last_clean":"2024-09-13T05:01:01.606515+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:01:01.606515+0000","last_undegraded":"2024-09-13T05:01:01.606515+0000","last_fullsized":"2024-09-13T05:01:01.606515+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'470","last_scrub_stamp":"2024-09-13T05:01:01.606449+0000","last_deep_scrub":"160'470","last_deep_scrub_stamp":"2024-09-13T05:01:01.606449+0000","last_clean_scrub_stamp":"2024-09-13T05:01:01.606449+0000","objects_scrubbed":14,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:57:08.407246+0000","scrub_duration":0.021121075999999999,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"160'644","reported_seq":2088,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:10.230566+0000","last_change":"2024-09-13T05:01:10.230869+0000","last_active":"2024-09-13T05:01:10.230566+0000","last_peered":"2024-09-13T05:01:10.230566+0000","last_clean":"2024-09-13T05:01:10.230566+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:01:10.230566+0000","last_undegraded":"2024-09-13T05:01:10.230566+0000","last_fullsized":"2024-09-13T05:01:10.230566+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'644","last_scrub_stamp":"2024-09-13T05:01:10.230510+0000","last_deep_scrub":"160'644","last_deep_scrub_stamp":"2024-09-13T05:01:10.230510+0000","last_clean_scrub_stamp":"2024-09-13T05:01:10.230510+0000","objects_scrubbed":14,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:02:31.739197+0000","scrub_duration":0.027995584,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18971,"ondisk_log_size":18971,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45272,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326568,"statfs":{"total":96636764160,"available":96590405632,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.81200000000000006,"15min":0.81499999999999995},"min":{"1min":0.49299999999999999,"5min":0.438,"15min":0.438},"max":{"1min":2.375,"5min":2.375,"15min":2.375},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86799999999999999,"15min":0.85399999999999998},"min":{"1min":0.48299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":1.004}]},{"osd":1,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.88500000000000001,"15min":0.84899999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.141,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.504,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.1579999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.93500000000000005,"15min":0.96199999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.92600000000000005,"15min":0.91000000000000003},"min":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8839999999999999,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.94999999999999996,"15min":0.94699999999999995},"min":{"1min":0.57499999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.5409999999999999,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.98899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.728,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.82399999999999995,"15min":0.82999999999999996},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.044,"15min":1.05},"min":{"1min":0.43099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.1899999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.374,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.375,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":1.216},{"interface":"front","average":{"1min":1.115,"5min":1.034,"15min":1.048},"min":{"1min":0.46999999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":1.024}]},{"osd":6,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.998,"15min":1.018},"min":{"1min":0.434,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4610000000000001,"5min":1.843,"15min":1.843},"last":1.268},{"interface":"front","average":{"1min":1.181,"5min":1.139,"15min":1.163},"min":{"1min":0.497,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.1439999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.2010000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.42199999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.82099999999999995}]},{"osd":1,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.755},"min":{"1min":0.38700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.202,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.44600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.218,"5min":1.337,"15min":1.337},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.91400000000000003,"15min":0.94799999999999995},"min":{"1min":0.43099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.8700000000000001,"5min":5.343,"15min":5.343},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.48499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.256}]},{"osd":3,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.88400000000000001,"15min":0.89300000000000002},"min":{"1min":0.499,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92000000000000004,"15min":0.93799999999999994},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.89700000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.90600000000000003,"15min":0.94299999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.103,"5min":1.323,"15min":1.323},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95399999999999996,"15min":0.97899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3220000000000001,"5min":1.575,"15min":1.575},"last":0.97499999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.93000000000000005,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6919999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.123},{"interface":"front","average":{"1min":1.056,"5min":0.97499999999999998,"15min":0.97299999999999998},"min":{"1min":0.53400000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.042}]},{"osd":7,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0149999999999999,"15min":1.0289999999999999},"min":{"1min":0.435,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.605,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.218},{"interface":"front","average":{"1min":1.03,"5min":0.98599999999999999,"15min":0.97699999999999998},"min":{"1min":0.59499999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5820000000000001,"5min":1.762,"15min":1.762},"last":1.377}]}]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51032,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320808,"statfs":{"total":96636764160,"available":96584507392,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86499999999999999,"15min":0.90600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.972,"5min":3.972,"15min":3.972},"last":0.754},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.52300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.80800000000000005}]},{"osd":1,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.84799999999999998},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.4730000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89300000000000002,"15min":0.91400000000000003},"min":{"1min":0.53200000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2809999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.83299999999999996}]},{"osd":2,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86199999999999999,"15min":0.879},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3109999999999999,"5min":1.607,"15min":1.607},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.88300000000000001,"15min":0.89500000000000002},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4350000000000001,"5min":1.579,"15min":1.579},"last":0.96599999999999997}]},{"osd":3,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":1.163,"15min":1.3149999999999999},"min":{"1min":0.53300000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1919999999999999,"5min":11.266,"15min":11.266},"last":1.006},{"interface":"front","average":{"1min":1.0880000000000001,"5min":0.96499999999999997,"15min":0.95899999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.9009999999999998,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.103}]},{"osd":5,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80100000000000005,"15min":0.81100000000000005},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.256,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.878,"15min":0.89000000000000001},"min":{"1min":0.36499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.3580000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1379999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88400000000000001,"15min":0.91200000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.181,"5min":1.325,"15min":1.325},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.98799999999999999,"15min":1.016},"min":{"1min":0.48099999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3640000000000001,"5min":1.752,"15min":1.752},"last":1.052}]},{"osd":7,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.97199999999999998,"15min":0.97399999999999998},"min":{"1min":0.48099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5229999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.0149999999999999,"15min":1.0429999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4199999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.028}]}]},{"osd":3,"up_from":15,"seq":64424509486,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.628},"min":{"1min":0.30199999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.94399999999999995,"5min":1.47,"15min":1.47},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.24099999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.3149999999999999,"5min":1.552,"15min":1.552},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.67900000000000005},"min":{"1min":0.32900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1419999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.625},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.76800000000000002,"15min":0.81000000000000005},"min":{"1min":0.34200000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1539999999999999,"5min":1.583,"15min":1.583},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.32000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.85599999999999998,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.79800000000000004},"min":{"1min":0.38900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3979999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.93600000000000005}]},{"osd":4,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.85699999999999998},"min":{"1min":0.63100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.25,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.63,"5min":0.432,"15min":0.432},"max":{"1min":1.1080000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.58299999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3779999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96099999999999997,"15min":0.995},"min":{"1min":0.69799999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.417,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.98399999999999999,"15min":1.002},"min":{"1min":0.64600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.335,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0289999999999999,"15min":1.0629999999999999},"min":{"1min":0.64200000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5609999999999999,"5min":1.994,"15min":1.994},"last":0.99199999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0109999999999999,"15min":1.044},"min":{"1min":0.66100000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.595,"5min":1.796,"15min":1.796},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.022,"5min":1.0029999999999999,"15min":1.0169999999999999},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3500000000000001,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.0469999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.33600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.93700000000000006,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.62},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.32000000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.2829999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.60099999999999998,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.223,"15min":0.223},"max":{"1min":1.8939999999999999,"5min":3.113,"15min":3.113},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.92600000000000005,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.86399999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.624,"15min":0.58999999999999997},"min":{"1min":0.442,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.70799999999999996,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9199999999999999,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":1.238}]},{"osd":4,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.87,"15min":0.83499999999999996},"min":{"1min":0.69999999999999996,"5min":0.374,"15min":0.374},"max":{"1min":2.0430000000000001,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.93000000000000005,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3560000000000001,"5min":1.861,"15min":1.861},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.71499999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4119999999999999,"5min":1.627,"15min":1.627},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.05,"5min":0.94799999999999995,"15min":0.92700000000000005},"min":{"1min":0.64600000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.9430000000000001,"5min":3.327,"15min":3.327},"last":1.157}]},{"osd":6,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.99199999999999999,"15min":1.0109999999999999},"min":{"1min":0.78500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.411,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.212},{"interface":"front","average":{"1min":1.046,"5min":0.96499999999999997,"15min":0.96099999999999997},"min":{"1min":0.65000000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.405,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.103}]},{"osd":7,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.006,"15min":1.02},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3819999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.268},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.66800000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4650000000000001,"5min":2.254,"15min":2.254},"last":1.129}]}]},{"osd":1,"up_from":15,"seq":64424509486,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59268,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312572,"statfs":{"total":96636764160,"available":96576073728,"internally_reserved":0,"allocated":1961984,"data_stored":1501653,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.315,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.1259999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.81899999999999995}]},{"osd":2,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.39000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.96399999999999997,"5min":1.407,"15min":1.407},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.208,"5min":1.401,"15min":1.401},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.42599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1000000000000001,"5min":1.306,"15min":1.306},"last":0.879}]},{"osd":4,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.56399999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.2270000000000001,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.079},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.88800000000000001,"15min":0.91900000000000004},"min":{"1min":0.51300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0509999999999999,"5min":1.655,"15min":1.655},"last":0.998}]},{"osd":5,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85199999999999998,"15min":0.86299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.153,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.62,"5min":0.434,"15min":0.434},"max":{"1min":1.1850000000000001,"5min":1.49,"15min":1.49},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.88900000000000001,"15min":0.90800000000000003},"min":{"1min":0.62,"5min":0.496,"15min":0.496},"max":{"1min":1.2250000000000001,"5min":1.577,"15min":1.577},"last":1.167},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.95299999999999996},"min":{"1min":0.65000000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3029999999999999,"5min":1.738,"15min":1.738},"last":0.90700000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.60499999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2290000000000001,"5min":1.766,"15min":1.766},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94799999999999995,"15min":0.95499999999999996},"min":{"1min":0.66600000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.349,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.123}]}]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56100000000000005,"15min":0.57999999999999996},"min":{"1min":0.26900000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.85799999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.628,"5min":1.175,"15min":1.4430000000000001},"min":{"1min":0.28399999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1319999999999999,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.82799999999999996}]},{"osd":2,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.752},{"interface":"front","average":{"1min":0.67800000000000005,"5min":1.2390000000000001,"15min":1.492},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.1799999999999999,"5min":15.802,"15min":15.802},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.65800000000000003,"15min":0.61799999999999999},"min":{"1min":0.35799999999999998,"5min":0.22,"15min":0.22},"max":{"1min":1.5569999999999999,"5min":2.915,"15min":2.915},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.751,"15min":0.77000000000000002},"min":{"1min":0.38400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.4019999999999999,"15min":1.663},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2350000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80300000000000005,"15min":0.77800000000000002},"min":{"1min":0.59599999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.353,"5min":3.109,"15min":3.109},"last":1.2549999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.268,"15min":1.411},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6679999999999999,"5min":11.474,"15min":11.474},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.3360000000000001,"15min":1.5189999999999999},"min":{"1min":0.61799999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.611,"5min":11.528,"15min":11.528},"last":1.0189999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.93799999999999994,"15min":0.94899999999999995},"min":{"1min":0.57799999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4330000000000001,"5min":1.571,"15min":1.571},"last":1.2190000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.498,"15min":1.764},"min":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5169999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":1.161}]},{"osd":7,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.542,"15min":1.802},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.617,"5min":15.834,"15min":15.834},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.95999999999999996,"15min":0.93100000000000005},"min":{"1min":0.61499999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.77,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.381}]}]},{"osd":5,"up_from":14,"seq":60129542194,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1978368,"data_stored":1501653,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.90900000000000003},"min":{"1min":0.48899999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2649999999999999,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.82699999999999996,"15min":0.80600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.97699999999999998}]},{"osd":1,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.77800000000000002,"15min":0.74099999999999999},"min":{"1min":0.46500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.75700000000000001,"15min":0.73499999999999999},"min":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.93999999999999995}]},{"osd":2,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.48,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.94999999999999996,"15min":0.996},"min":{"1min":0.65600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.107,"5min":4.633,"15min":4.633},"last":1.016}]},{"osd":3,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.89300000000000002,"15min":0.85599999999999998},"min":{"1min":0.64300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.876,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.135,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89700000000000002,"15min":0.93500000000000005},"min":{"1min":0.45300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.204,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.88600000000000001,"15min":0.85999999999999999},"min":{"1min":0.60499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.3089999999999999,"5min":1.409,"15min":1.409},"last":1.194}]},{"osd":6,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4159999999999999,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.099,"15min":1.113},"min":{"1min":0.66300000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.601,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.2410000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.97499999999999998,"15min":0.94799999999999995},"min":{"1min":0.53200000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3859999999999999,"5min":1.454,"15min":1.454},"last":1.151},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95099999999999996,"15min":0.91300000000000003},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.554,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.0600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:01:19.091 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:01:19.092 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:01:19.092 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:01:19.092 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T04:58:03.193961+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=4, tm_min=58, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=0, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:01:19.093 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T05:01:39.094 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:01:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-13T05:01:39.413 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-13T05:01:39.432 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-13T05:01:38.968563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":493,"num_read":163161,"num_read_kb":654690,"num_write":21603,"num_write_kb":344301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1894,"num_bytes_recovered":73793959,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19151,"ondisk_log_size":19151,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":413232,"kb_used_data":15088,"kb_used_omap":12,"kb_used_meta":398067,"kb_avail":754561488,"statfs":{"total":773094113280,"available":772670963712,"internally_reserved":0,"allocated":15450112,"data_stored":8470940,"data_compressed":13317,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12743,"internal_metadata":407621177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003640"},"pg_stats":[{"pgid":"3.1f","version":"160'623","reported_seq":3953,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:18.762267+0000","last_change":"2024-09-13T05:01:18.762491+0000","last_active":"2024-09-13T05:01:18.762267+0000","last_peered":"2024-09-13T05:01:18.762267+0000","last_clean":"2024-09-13T05:01:18.762267+0000","last_became_active":"2024-09-13T05:00:20.959350+0000","last_became_peered":"2024-09-13T05:00:20.959350+0000","last_unstale":"2024-09-13T05:01:18.762267+0000","last_undegraded":"2024-09-13T05:01:18.762267+0000","last_fullsized":"2024-09-13T05:01:18.762267+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'623","last_scrub_stamp":"2024-09-13T05:01:18.762213+0000","last_deep_scrub":"160'623","last_deep_scrub_stamp":"2024-09-13T05:01:18.762213+0000","last_clean_scrub_stamp":"2024-09-13T05:01:18.762213+0000","objects_scrubbed":14,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:44.592232+0000","scrub_duration":0.022273035,"objects_trimmed":0,"snaptrim_duration":0.0018301820000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4453,"num_read_kb":16334,"num_write":607,"num_write_kb":9819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":8122229,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"160'398","reported_seq":2294,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:07.726244+0000","last_change":"2024-09-13T05:01:07.726497+0000","last_active":"2024-09-13T05:01:07.726244+0000","last_peered":"2024-09-13T05:01:07.726244+0000","last_clean":"2024-09-13T05:01:07.726244+0000","last_became_active":"2024-09-13T04:58:03.927925+0000","last_became_peered":"2024-09-13T04:58:03.927925+0000","last_unstale":"2024-09-13T05:01:07.726244+0000","last_undegraded":"2024-09-13T05:01:07.726244+0000","last_fullsized":"2024-09-13T05:01:07.726244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'398","last_scrub_stamp":"2024-09-13T05:01:07.726192+0000","last_deep_scrub":"160'398","last_deep_scrub_stamp":"2024-09-13T05:01:07.726192+0000","last_clean_scrub_stamp":"2024-09-13T05:01:07.726192+0000","objects_scrubbed":14,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:27:23.349006+0000","scrub_duration":0.020502889,"objects_trimmed":0,"snaptrim_duration":0.0017125829999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2757,"num_read_kb":18338,"num_write":336,"num_write_kb":8824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"160'552","reported_seq":3843,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:18.261477+0000","last_change":"2024-09-13T05:01:18.261743+0000","last_active":"2024-09-13T05:01:18.261477+0000","last_peered":"2024-09-13T05:01:18.261477+0000","last_clean":"2024-09-13T05:01:18.261477+0000","last_became_active":"2024-09-13T04:58:03.917505+0000","last_became_peered":"2024-09-13T04:58:03.917505+0000","last_unstale":"2024-09-13T05:01:18.261477+0000","last_undegraded":"2024-09-13T05:01:18.261477+0000","last_fullsized":"2024-09-13T05:01:18.261477+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'552","last_scrub_stamp":"2024-09-13T05:01:18.261425+0000","last_deep_scrub":"160'552","last_deep_scrub_stamp":"2024-09-13T05:01:18.261425+0000","last_clean_scrub_stamp":"2024-09-13T05:01:18.261425+0000","objects_scrubbed":17,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:27:44.424463+0000","scrub_duration":0.032893932000000001,"objects_trimmed":0,"snaptrim_duration":0.0022345809999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4168,"num_read_kb":23676,"num_write":661,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"160'621","reported_seq":2293,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:59.599124+0000","last_change":"2024-09-13T05:00:59.599361+0000","last_active":"2024-09-13T05:00:59.599124+0000","last_peered":"2024-09-13T05:00:59.599124+0000","last_clean":"2024-09-13T05:00:59.599124+0000","last_became_active":"2024-09-13T05:00:20.960432+0000","last_became_peered":"2024-09-13T05:00:20.960432+0000","last_unstale":"2024-09-13T05:00:59.599124+0000","last_undegraded":"2024-09-13T05:00:59.599124+0000","last_fullsized":"2024-09-13T05:00:59.599124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'621","last_scrub_stamp":"2024-09-13T05:00:59.599071+0000","last_deep_scrub":"160'621","last_deep_scrub_stamp":"2024-09-13T05:00:59.599071+0000","last_clean_scrub_stamp":"2024-09-13T05:00:59.599071+0000","objects_scrubbed":17,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:48.833303+0000","scrub_duration":0.031728854000000001,"objects_trimmed":0,"snaptrim_duration":0.001482602,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":2169,"num_read_kb":8254,"num_write":741,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":5780877,"num_keys_recovered":68,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"160'554","reported_seq":5259,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:18.252931+0000","last_change":"2024-09-13T05:01:18.253233+0000","last_active":"2024-09-13T05:01:18.252931+0000","last_peered":"2024-09-13T05:01:18.252931+0000","last_clean":"2024-09-13T05:01:18.252931+0000","last_became_active":"2024-09-13T04:58:03.920102+0000","last_became_peered":"2024-09-13T04:58:03.920102+0000","last_unstale":"2024-09-13T05:01:18.252931+0000","last_undegraded":"2024-09-13T05:01:18.252931+0000","last_fullsized":"2024-09-13T05:01:18.252931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'554","last_scrub_stamp":"2024-09-13T05:01:18.252881+0000","last_deep_scrub":"160'554","last_deep_scrub_stamp":"2024-09-13T05:01:18.252881+0000","last_clean_scrub_stamp":"2024-09-13T05:01:18.252881+0000","objects_scrubbed":10,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:48.904271+0000","scrub_duration":0.019161167,"objects_trimmed":0,"snaptrim_duration":0.0023960190000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5861,"num_read_kb":17789,"num_write":576,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"160'518","reported_seq":4017,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:03.585059+0000","last_change":"2024-09-13T05:01:03.585288+0000","last_active":"2024-09-13T05:01:03.585059+0000","last_peered":"2024-09-13T05:01:03.585059+0000","last_clean":"2024-09-13T05:01:03.585059+0000","last_became_active":"2024-09-13T05:00:20.949859+0000","last_became_peered":"2024-09-13T05:00:20.949859+0000","last_unstale":"2024-09-13T05:01:03.585059+0000","last_undegraded":"2024-09-13T05:01:03.585059+0000","last_fullsized":"2024-09-13T05:01:03.585059+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'518","last_scrub_stamp":"2024-09-13T05:01:03.585006+0000","last_deep_scrub":"160'518","last_deep_scrub_stamp":"2024-09-13T05:01:03.585006+0000","last_clean_scrub_stamp":"2024-09-13T05:01:03.585006+0000","objects_scrubbed":14,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:00:43.454127+0000","scrub_duration":0.026602105000000001,"objects_trimmed":0,"snaptrim_duration":0.0028406569999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":4761,"num_read_kb":13803,"num_write":544,"num_write_kb":11712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":7140863,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"160'519","reported_seq":19504,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:16.240834+0000","last_change":"2024-09-13T05:01:16.241169+0000","last_active":"2024-09-13T05:01:16.240834+0000","last_peered":"2024-09-13T05:01:16.240834+0000","last_clean":"2024-09-13T05:01:16.240834+0000","last_became_active":"2024-09-13T05:00:19.956063+0000","last_became_peered":"2024-09-13T05:00:19.956063+0000","last_unstale":"2024-09-13T05:01:16.240834+0000","last_undegraded":"2024-09-13T05:01:16.240834+0000","last_fullsized":"2024-09-13T05:01:16.240834+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'519","last_scrub_stamp":"2024-09-13T05:01:16.240778+0000","last_deep_scrub":"160'519","last_deep_scrub_stamp":"2024-09-13T05:01:16.240778+0000","last_clean_scrub_stamp":"2024-09-13T05:01:16.240778+0000","objects_scrubbed":17,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:08:41.139301+0000","scrub_duration":0.031141914999999999,"objects_trimmed":0,"snaptrim_duration":0.0024416889999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":19395,"num_read_kb":96246,"num_write":521,"num_write_kb":19281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":17095156,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"160'496","reported_seq":3105,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:05.754297+0000","last_change":"2024-09-13T05:01:05.754580+0000","last_active":"2024-09-13T05:01:05.754297+0000","last_peered":"2024-09-13T05:01:05.754297+0000","last_clean":"2024-09-13T05:01:05.754297+0000","last_became_active":"2024-09-13T04:58:03.928959+0000","last_became_peered":"2024-09-13T04:58:03.928959+0000","last_unstale":"2024-09-13T05:01:05.754297+0000","last_undegraded":"2024-09-13T05:01:05.754297+0000","last_fullsized":"2024-09-13T05:01:05.754297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'496","last_scrub_stamp":"2024-09-13T05:01:05.754245+0000","last_deep_scrub":"160'496","last_deep_scrub_stamp":"2024-09-13T05:01:05.754245+0000","last_clean_scrub_stamp":"2024-09-13T05:01:05.754245+0000","objects_scrubbed":14,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:18:53.100903+0000","scrub_duration":0.024650089,"objects_trimmed":0,"snaptrim_duration":0.0014982979999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3261,"num_read_kb":9834,"num_write":552,"num_write_kb":6010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"160'348","reported_seq":3805,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:11.702285+0000","last_change":"2024-09-13T05:01:11.702577+0000","last_active":"2024-09-13T05:01:11.702285+0000","last_peered":"2024-09-13T05:01:11.702285+0000","last_clean":"2024-09-13T05:01:11.702285+0000","last_became_active":"2024-09-13T04:58:03.933822+0000","last_became_peered":"2024-09-13T04:58:03.933822+0000","last_unstale":"2024-09-13T05:01:11.702285+0000","last_undegraded":"2024-09-13T05:01:11.702285+0000","last_fullsized":"2024-09-13T05:01:11.702285+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'348","last_scrub_stamp":"2024-09-13T05:01:11.702213+0000","last_deep_scrub":"160'348","last_deep_scrub_stamp":"2024-09-13T05:01:11.702213+0000","last_clean_scrub_stamp":"2024-09-13T05:01:11.702213+0000","objects_scrubbed":16,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:57:36.853328+0000","scrub_duration":0.02272708,"objects_trimmed":0,"snaptrim_duration":0.0020482080000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3215,"num_read_kb":21670,"num_write":307,"num_write_kb":14362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"160'570","reported_seq":2654,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:00.438428+0000","last_change":"2024-09-13T05:01:00.438707+0000","last_active":"2024-09-13T05:01:00.438428+0000","last_peered":"2024-09-13T05:01:00.438428+0000","last_clean":"2024-09-13T05:01:00.438428+0000","last_became_active":"2024-09-13T04:58:03.919560+0000","last_became_peered":"2024-09-13T04:58:03.919560+0000","last_unstale":"2024-09-13T05:01:00.438428+0000","last_undegraded":"2024-09-13T05:01:00.438428+0000","last_fullsized":"2024-09-13T05:01:00.438428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'570","last_scrub_stamp":"2024-09-13T05:01:00.438366+0000","last_deep_scrub":"160'570","last_deep_scrub_stamp":"2024-09-13T05:01:00.438366+0000","last_clean_scrub_stamp":"2024-09-13T05:01:00.438366+0000","objects_scrubbed":10,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:43:38.046169+0000","scrub_duration":0.0221271,"objects_trimmed":0,"snaptrim_duration":0.001248627,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3529,"num_read_kb":19855,"num_write":675,"num_write_kb":7279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"160'440","reported_seq":1302,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:03.856773+0000","last_change":"2024-09-13T05:01:03.857080+0000","last_active":"2024-09-13T05:01:03.856773+0000","last_peered":"2024-09-13T05:01:03.856773+0000","last_clean":"2024-09-13T05:01:03.856773+0000","last_became_active":"2024-09-13T04:58:03.925069+0000","last_became_peered":"2024-09-13T04:58:03.925069+0000","last_unstale":"2024-09-13T05:01:03.856773+0000","last_undegraded":"2024-09-13T05:01:03.856773+0000","last_fullsized":"2024-09-13T05:01:03.856773+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'440","last_scrub_stamp":"2024-09-13T05:01:03.856711+0000","last_deep_scrub":"160'440","last_deep_scrub_stamp":"2024-09-13T05:01:03.856711+0000","last_clean_scrub_stamp":"2024-09-13T05:01:03.856711+0000","objects_scrubbed":9,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:14.107366+0000","scrub_duration":0.016563472999999999,"objects_trimmed":0,"snaptrim_duration":0.003521265,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1553,"num_read_kb":10676,"num_write":460,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"160'760","reported_seq":4445,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:14.245661+0000","last_change":"2024-09-13T05:01:14.245987+0000","last_active":"2024-09-13T05:01:14.245661+0000","last_peered":"2024-09-13T05:01:14.245661+0000","last_clean":"2024-09-13T05:01:14.245661+0000","last_became_active":"2024-09-13T04:58:03.915311+0000","last_became_peered":"2024-09-13T04:58:03.915311+0000","last_unstale":"2024-09-13T05:01:14.245661+0000","last_undegraded":"2024-09-13T05:01:14.245661+0000","last_fullsized":"2024-09-13T05:01:14.245661+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'760","last_scrub_stamp":"2024-09-13T05:01:14.245607+0000","last_deep_scrub":"160'760","last_deep_scrub_stamp":"2024-09-13T05:01:14.245607+0000","last_clean_scrub_stamp":"2024-09-13T05:01:14.245607+0000","objects_scrubbed":16,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:53:23.500615+0000","scrub_duration":0.031691538999999998,"objects_trimmed":0,"snaptrim_duration":0.001776581,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5611,"num_read_kb":13680,"num_write":945,"num_write_kb":5074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:02.240728+0000","last_change":"2024-09-13T05:01:02.240954+0000","last_active":"2024-09-13T05:01:02.240728+0000","last_peered":"2024-09-13T05:01:02.240728+0000","last_clean":"2024-09-13T05:01:02.240728+0000","last_became_active":"2024-09-13T04:57:55.148616+0000","last_became_peered":"2024-09-13T04:57:55.148616+0000","last_unstale":"2024-09-13T05:01:02.240728+0000","last_undegraded":"2024-09-13T05:01:02.240728+0000","last_fullsized":"2024-09-13T05:01:02.240728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:02.240675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:02.240675+0000","last_clean_scrub_stamp":"2024-09-13T05:01:02.240675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:57:03.447335+0000","scrub_duration":0.0029906199999999998,"objects_trimmed":0,"snaptrim_duration":0.0022035000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"160'356","reported_seq":2715,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:06.563872+0000","last_change":"2024-09-13T05:01:06.564169+0000","last_active":"2024-09-13T05:01:06.563872+0000","last_peered":"2024-09-13T05:01:06.563872+0000","last_clean":"2024-09-13T05:01:06.563872+0000","last_became_active":"2024-09-13T04:58:19.308811+0000","last_became_peered":"2024-09-13T04:58:19.308811+0000","last_unstale":"2024-09-13T05:01:06.563872+0000","last_undegraded":"2024-09-13T05:01:06.563872+0000","last_fullsized":"2024-09-13T05:01:06.563872+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'356","last_scrub_stamp":"2024-09-13T05:01:06.563812+0000","last_deep_scrub":"160'356","last_deep_scrub_stamp":"2024-09-13T05:01:06.563812+0000","last_clean_scrub_stamp":"2024-09-13T05:01:06.563812+0000","objects_scrubbed":17,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:19:27.190719+0000","scrub_duration":0.028154248,"objects_trimmed":0,"snaptrim_duration":0.001741476,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2154,"num_read_kb":23093,"num_write":285,"num_write_kb":19869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4992010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:59.333515+0000","last_change":"2024-09-13T05:00:59.333741+0000","last_active":"2024-09-13T05:00:59.333515+0000","last_peered":"2024-09-13T05:00:59.333515+0000","last_clean":"2024-09-13T05:00:59.333515+0000","last_became_active":"2024-09-13T04:57:55.148249+0000","last_became_peered":"2024-09-13T04:57:55.148249+0000","last_unstale":"2024-09-13T05:00:59.333515+0000","last_undegraded":"2024-09-13T05:00:59.333515+0000","last_fullsized":"2024-09-13T05:00:59.333515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:00:59.333468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:00:59.333468+0000","last_clean_scrub_stamp":"2024-09-13T05:00:59.333468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:38:12.476332+0000","scrub_duration":0.0032761819999999999,"objects_trimmed":0,"snaptrim_duration":0.00068990100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"160'761","reported_seq":2267,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:30.277299+0000","last_change":"2024-09-13T05:01:30.277567+0000","last_active":"2024-09-13T05:01:30.277299+0000","last_peered":"2024-09-13T05:01:30.277299+0000","last_clean":"2024-09-13T05:01:30.277299+0000","last_became_active":"2024-09-13T04:58:03.924859+0000","last_became_peered":"2024-09-13T04:58:03.924859+0000","last_unstale":"2024-09-13T05:01:30.277299+0000","last_undegraded":"2024-09-13T05:01:30.277299+0000","last_fullsized":"2024-09-13T05:01:30.277299+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'761","last_scrub_stamp":"2024-09-13T05:01:30.277247+0000","last_deep_scrub":"160'761","last_deep_scrub_stamp":"2024-09-13T05:01:30.277247+0000","last_clean_scrub_stamp":"2024-09-13T05:01:30.277247+0000","objects_scrubbed":15,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:41.620901+0000","scrub_duration":0.033614231000000001,"objects_trimmed":0,"snaptrim_duration":0.0023966859999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4458,"num_read_kb":12594,"num_write":1002,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:03.206685+0000","last_change":"2024-09-13T05:01:03.206910+0000","last_active":"2024-09-13T05:01:03.206685+0000","last_peered":"2024-09-13T05:01:03.206685+0000","last_clean":"2024-09-13T05:01:03.206685+0000","last_became_active":"2024-09-13T04:57:54.152007+0000","last_became_peered":"2024-09-13T04:57:54.152007+0000","last_unstale":"2024-09-13T05:01:03.206685+0000","last_undegraded":"2024-09-13T05:01:03.206685+0000","last_fullsized":"2024-09-13T05:01:03.206685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:03.206631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:03.206631+0000","last_clean_scrub_stamp":"2024-09-13T05:01:03.206631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:48:34.855796+0000","scrub_duration":0.0032981880000000001,"objects_trimmed":0,"snaptrim_duration":0.0014891220000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"160'658","reported_seq":1810,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:11.658844+0000","last_change":"2024-09-13T05:01:11.659058+0000","last_active":"2024-09-13T05:01:11.658844+0000","last_peered":"2024-09-13T05:01:11.658844+0000","last_clean":"2024-09-13T05:01:11.658844+0000","last_became_active":"2024-09-13T05:00:42.127080+0000","last_became_peered":"2024-09-13T05:00:42.127080+0000","last_unstale":"2024-09-13T05:01:11.658844+0000","last_undegraded":"2024-09-13T05:01:11.658844+0000","last_fullsized":"2024-09-13T05:01:11.658844+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'658","last_scrub_stamp":"2024-09-13T05:01:11.658787+0000","last_deep_scrub":"160'658","last_deep_scrub_stamp":"2024-09-13T05:01:11.658787+0000","last_clean_scrub_stamp":"2024-09-13T05:01:11.658787+0000","objects_scrubbed":25,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:37:52.619580+0000","scrub_duration":0.045409115,"objects_trimmed":6,"snaptrim_duration":0.0030854749999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":3117,"num_read_kb":14959,"num_write":800,"num_write_kb":8991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.589479+0000","last_change":"2024-09-13T05:00:57.589690+0000","last_active":"2024-09-13T05:00:57.589479+0000","last_peered":"2024-09-13T05:00:57.589479+0000","last_clean":"2024-09-13T05:00:57.589479+0000","last_became_active":"2024-09-13T04:57:54.375605+0000","last_became_peered":"2024-09-13T04:57:54.375605+0000","last_unstale":"2024-09-13T05:00:57.589479+0000","last_undegraded":"2024-09-13T05:00:57.589479+0000","last_fullsized":"2024-09-13T05:00:57.589479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:00:57.589425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:00:57.589425+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.589425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:25:38.515636+0000","scrub_duration":0.002791155,"objects_trimmed":0,"snaptrim_duration":0.00077885400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"160'575","reported_seq":2480,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:05.417896+0000","last_change":"2024-09-13T05:01:05.418152+0000","last_active":"2024-09-13T05:01:05.417896+0000","last_peered":"2024-09-13T05:01:05.417896+0000","last_clean":"2024-09-13T05:01:05.417896+0000","last_became_active":"2024-09-13T04:58:03.923952+0000","last_became_peered":"2024-09-13T04:58:03.923952+0000","last_unstale":"2024-09-13T05:01:05.417896+0000","last_undegraded":"2024-09-13T05:01:05.417896+0000","last_fullsized":"2024-09-13T05:01:05.417896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'575","last_scrub_stamp":"2024-09-13T05:01:05.417814+0000","last_deep_scrub":"160'575","last_deep_scrub_stamp":"2024-09-13T05:01:05.417814+0000","last_clean_scrub_stamp":"2024-09-13T05:01:05.417814+0000","objects_scrubbed":17,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:34:21.482343+0000","scrub_duration":0.027707463000000002,"objects_trimmed":0,"snaptrim_duration":0.001156965,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3850,"num_read_kb":10839,"num_write":590,"num_write_kb":6337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"160'942","reported_seq":4980,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.009814+0000","last_change":"2024-09-13T05:00:57.009937+0000","last_active":"2024-09-13T05:00:57.009814+0000","last_peered":"2024-09-13T05:00:57.009814+0000","last_clean":"2024-09-13T05:00:57.009814+0000","last_became_active":"2024-09-13T04:58:03.914224+0000","last_became_peered":"2024-09-13T04:58:03.914224+0000","last_unstale":"2024-09-13T05:00:57.009814+0000","last_undegraded":"2024-09-13T05:00:57.009814+0000","last_fullsized":"2024-09-13T05:00:57.009814+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'942","last_scrub_stamp":"2024-09-13T05:00:57.009788+0000","last_deep_scrub":"160'942","last_deep_scrub_stamp":"2024-09-13T05:00:57.009788+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.009788+0000","objects_scrubbed":17,"log_size":942,"log_dups_size":0,"ondisk_log_size":942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:37:03.234330+0000","scrub_duration":0.015432392,"objects_trimmed":0,"snaptrim_duration":0.0013546750000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7212,"num_read_kb":22791,"num_write":981,"num_write_kb":15695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"160'3","reported_seq":169,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.246290+0000","last_change":"2024-09-13T05:00:57.246365+0000","last_active":"2024-09-13T05:00:57.246290+0000","last_peered":"2024-09-13T05:00:57.246290+0000","last_clean":"2024-09-13T05:00:57.246290+0000","last_became_active":"2024-09-13T04:57:54.375879+0000","last_became_peered":"2024-09-13T04:57:54.375879+0000","last_unstale":"2024-09-13T05:00:57.246290+0000","last_undegraded":"2024-09-13T05:00:57.246290+0000","last_fullsized":"2024-09-13T05:00:57.246290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'3","last_scrub_stamp":"2024-09-13T05:00:57.246270+0000","last_deep_scrub":"160'3","last_deep_scrub_stamp":"2024-09-13T05:00:57.246270+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.246270+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:09.860854+0000","scrub_duration":0.0041561840000000003,"objects_trimmed":0,"snaptrim_duration":0.00093438600000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"160'1238","reported_seq":2930,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:14.745766+0000","last_change":"2024-09-13T05:01:14.746036+0000","last_active":"2024-09-13T05:01:14.745766+0000","last_peered":"2024-09-13T05:01:14.745766+0000","last_clean":"2024-09-13T05:01:14.745766+0000","last_became_active":"2024-09-13T04:58:03.935380+0000","last_became_peered":"2024-09-13T04:58:03.935380+0000","last_unstale":"2024-09-13T05:01:14.745766+0000","last_undegraded":"2024-09-13T05:01:14.745766+0000","last_fullsized":"2024-09-13T05:01:14.745766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1238","last_scrub_stamp":"2024-09-13T05:01:14.745708+0000","last_deep_scrub":"160'1238","last_deep_scrub_stamp":"2024-09-13T05:01:14.745708+0000","last_clean_scrub_stamp":"2024-09-13T05:01:14.745708+0000","objects_scrubbed":21,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:16:16.291852+0000","scrub_duration":0.044443238000000003,"objects_trimmed":0,"snaptrim_duration":0.001195827,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":7044,"num_read_kb":20823,"num_write":1836,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":153,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:59.172290+0000","last_change":"2024-09-13T05:00:59.172486+0000","last_active":"2024-09-13T05:00:59.172290+0000","last_peered":"2024-09-13T05:00:59.172290+0000","last_clean":"2024-09-13T05:00:59.172290+0000","last_became_active":"2024-09-13T05:00:45.152844+0000","last_became_peered":"2024-09-13T05:00:45.152844+0000","last_unstale":"2024-09-13T05:00:59.172290+0000","last_undegraded":"2024-09-13T05:00:59.172290+0000","last_fullsized":"2024-09-13T05:00:59.172290+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:00:59.172238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:00:59.172238+0000","last_clean_scrub_stamp":"2024-09-13T05:00:59.172238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:16:00.503765+0000","scrub_duration":0.002972592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"160'465","reported_seq":5185,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:03.569553+0000","last_change":"2024-09-13T05:01:03.569863+0000","last_active":"2024-09-13T05:01:03.569553+0000","last_peered":"2024-09-13T05:01:03.569553+0000","last_clean":"2024-09-13T05:01:03.569553+0000","last_became_active":"2024-09-13T04:58:03.924116+0000","last_became_peered":"2024-09-13T04:58:03.924116+0000","last_unstale":"2024-09-13T05:01:03.569553+0000","last_undegraded":"2024-09-13T05:01:03.569553+0000","last_fullsized":"2024-09-13T05:01:03.569553+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'465","last_scrub_stamp":"2024-09-13T05:01:03.569497+0000","last_deep_scrub":"160'465","last_deep_scrub_stamp":"2024-09-13T05:01:03.569497+0000","last_clean_scrub_stamp":"2024-09-13T05:01:03.569497+0000","objects_scrubbed":11,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:50.070105+0000","scrub_duration":0.019109181999999999,"objects_trimmed":0,"snaptrim_duration":0.00072872200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5759,"num_read_kb":23227,"num_write":426,"num_write_kb":9859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:05.202580+0000","last_change":"2024-09-13T05:01:05.202792+0000","last_active":"2024-09-13T05:01:05.202580+0000","last_peered":"2024-09-13T05:01:05.202580+0000","last_clean":"2024-09-13T05:01:05.202580+0000","last_became_active":"2024-09-13T04:57:54.161091+0000","last_became_peered":"2024-09-13T04:57:54.161091+0000","last_unstale":"2024-09-13T05:01:05.202580+0000","last_undegraded":"2024-09-13T05:01:05.202580+0000","last_fullsized":"2024-09-13T05:01:05.202580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:05.202530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:05.202530+0000","last_clean_scrub_stamp":"2024-09-13T05:01:05.202530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:33:06.154178+0000","scrub_duration":0.0035771919999999999,"objects_trimmed":0,"snaptrim_duration":0.00094214600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"160'1058","reported_seq":3375,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:17.278641+0000","last_change":"2024-09-13T05:01:17.278877+0000","last_active":"2024-09-13T05:01:17.278641+0000","last_peered":"2024-09-13T05:01:17.278641+0000","last_clean":"2024-09-13T05:01:17.278641+0000","last_became_active":"2024-09-13T04:58:03.914022+0000","last_became_peered":"2024-09-13T04:58:03.914022+0000","last_unstale":"2024-09-13T05:01:17.278641+0000","last_undegraded":"2024-09-13T05:01:17.278641+0000","last_fullsized":"2024-09-13T05:01:17.278641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'1058","last_scrub_stamp":"2024-09-13T05:01:17.278588+0000","last_deep_scrub":"160'1058","last_deep_scrub_stamp":"2024-09-13T05:01:17.278588+0000","last_clean_scrub_stamp":"2024-09-13T05:01:17.278588+0000","objects_scrubbed":24,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:02.679032+0000","scrub_duration":0.038963978000000003,"objects_trimmed":0,"snaptrim_duration":0.001816518,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6902,"num_read_kb":25001,"num_write":1439,"num_write_kb":11491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"160'2","reported_seq":168,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:06.202282+0000","last_change":"2024-09-13T05:01:06.202514+0000","last_active":"2024-09-13T05:01:06.202282+0000","last_peered":"2024-09-13T05:01:06.202282+0000","last_clean":"2024-09-13T05:01:06.202282+0000","last_became_active":"2024-09-13T04:57:54.152655+0000","last_became_peered":"2024-09-13T04:57:54.152655+0000","last_unstale":"2024-09-13T05:01:06.202282+0000","last_undegraded":"2024-09-13T05:01:06.202282+0000","last_fullsized":"2024-09-13T05:01:06.202282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'2","last_scrub_stamp":"2024-09-13T05:01:06.202195+0000","last_deep_scrub":"160'2","last_deep_scrub_stamp":"2024-09-13T05:01:06.202195+0000","last_clean_scrub_stamp":"2024-09-13T05:01:06.202195+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:13:10.718476+0000","scrub_duration":0.0061865330000000001,"objects_trimmed":0,"snaptrim_duration":0.00178202,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"160'119","reported_seq":213,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:56.548750+0000","last_change":"2024-09-13T05:00:56.548894+0000","last_active":"2024-09-13T05:00:56.548750+0000","last_peered":"2024-09-13T05:00:56.548750+0000","last_clean":"2024-09-13T05:00:56.548750+0000","last_became_active":"2024-09-13T05:00:19.957956+0000","last_became_peered":"2024-09-13T05:00:19.957956+0000","last_unstale":"2024-09-13T05:00:56.548750+0000","last_undegraded":"2024-09-13T05:00:56.548750+0000","last_fullsized":"2024-09-13T05:00:56.548750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-09-13T05:00:56.548706+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-09-13T05:00:56.548706+0000","last_clean_scrub_stamp":"2024-09-13T05:00:56.548706+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:56:22.004941+0000","scrub_duration":0.011268804,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"160'537","reported_seq":3983,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:14.168194+0000","last_change":"2024-09-13T05:01:14.168472+0000","last_active":"2024-09-13T05:01:14.168194+0000","last_peered":"2024-09-13T05:01:14.168194+0000","last_clean":"2024-09-13T05:01:14.168194+0000","last_became_active":"2024-09-13T04:58:03.929192+0000","last_became_peered":"2024-09-13T04:58:03.929192+0000","last_unstale":"2024-09-13T05:01:14.168194+0000","last_undegraded":"2024-09-13T05:01:14.168194+0000","last_fullsized":"2024-09-13T05:01:14.168194+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'537","last_scrub_stamp":"2024-09-13T05:01:14.168137+0000","last_deep_scrub":"160'537","last_deep_scrub_stamp":"2024-09-13T05:01:14.168137+0000","last_clean_scrub_stamp":"2024-09-13T05:01:14.168137+0000","objects_scrubbed":17,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:48:49.345993+0000","scrub_duration":0.030468004,"objects_trimmed":0,"snaptrim_duration":0.0018778480000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4936,"num_read_kb":13307,"num_write":534,"num_write_kb":10377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"160'640","reported_seq":6140,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:13.190342+0000","last_change":"2024-09-13T05:01:13.190585+0000","last_active":"2024-09-13T05:01:13.190342+0000","last_peered":"2024-09-13T05:01:13.190342+0000","last_clean":"2024-09-13T05:01:13.190342+0000","last_became_active":"2024-09-13T04:58:03.915445+0000","last_became_peered":"2024-09-13T04:58:03.915445+0000","last_unstale":"2024-09-13T05:01:13.190342+0000","last_undegraded":"2024-09-13T05:01:13.190342+0000","last_fullsized":"2024-09-13T05:01:13.190342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'640","last_scrub_stamp":"2024-09-13T05:01:13.190281+0000","last_deep_scrub":"160'640","last_deep_scrub_stamp":"2024-09-13T05:01:13.190281+0000","last_clean_scrub_stamp":"2024-09-13T05:01:13.190281+0000","objects_scrubbed":13,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:58.704336+0000","scrub_duration":0.022449542999999999,"objects_trimmed":0,"snaptrim_duration":0.00073953899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6997,"num_read_kb":22524,"num_write":729,"num_write_kb":12031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"160'506","reported_seq":7345,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:11.196208+0000","last_change":"2024-09-13T05:01:11.196470+0000","last_active":"2024-09-13T05:01:11.196208+0000","last_peered":"2024-09-13T05:01:11.196208+0000","last_clean":"2024-09-13T05:01:11.196208+0000","last_became_active":"2024-09-13T05:00:20.961013+0000","last_became_peered":"2024-09-13T05:00:20.961013+0000","last_unstale":"2024-09-13T05:01:11.196208+0000","last_undegraded":"2024-09-13T05:01:11.196208+0000","last_fullsized":"2024-09-13T05:01:11.196208+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'506","last_scrub_stamp":"2024-09-13T05:01:11.196152+0000","last_deep_scrub":"160'506","last_deep_scrub_stamp":"2024-09-13T05:01:11.196152+0000","last_clean_scrub_stamp":"2024-09-13T05:01:11.196152+0000","objects_scrubbed":11,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:47.070920+0000","scrub_duration":0.022952017000000002,"objects_trimmed":0,"snaptrim_duration":0.0017090899999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":6390,"num_read_kb":24156,"num_write":476,"num_write_kb":14047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":8233165,"num_keys_recovered":21,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"160'343","reported_seq":2180,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:02.124222+0000","last_change":"2024-09-13T05:01:02.124450+0000","last_active":"2024-09-13T05:01:02.124222+0000","last_peered":"2024-09-13T05:01:02.124222+0000","last_clean":"2024-09-13T05:01:02.124222+0000","last_became_active":"2024-09-13T05:00:45.157021+0000","last_became_peered":"2024-09-13T05:00:45.157021+0000","last_unstale":"2024-09-13T05:01:02.124222+0000","last_undegraded":"2024-09-13T05:01:02.124222+0000","last_fullsized":"2024-09-13T05:01:02.124222+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'343","last_scrub_stamp":"2024-09-13T05:01:02.124166+0000","last_deep_scrub":"160'343","last_deep_scrub_stamp":"2024-09-13T05:01:02.124166+0000","last_clean_scrub_stamp":"2024-09-13T05:01:02.124166+0000","objects_scrubbed":13,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:01:01.825237+0000","scrub_duration":0.022449429999999999,"objects_trimmed":0,"snaptrim_duration":0.00086464899999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":1666,"num_read_kb":17145,"num_write":305,"num_write_kb":10162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"160'715","reported_seq":1969,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:06.920739+0000","last_change":"2024-09-13T05:01:06.920974+0000","last_active":"2024-09-13T05:01:06.920739+0000","last_peered":"2024-09-13T05:01:06.920739+0000","last_clean":"2024-09-13T05:01:06.920739+0000","last_became_active":"2024-09-13T04:58:03.924029+0000","last_became_peered":"2024-09-13T04:58:03.924029+0000","last_unstale":"2024-09-13T05:01:06.920739+0000","last_undegraded":"2024-09-13T05:01:06.920739+0000","last_fullsized":"2024-09-13T05:01:06.920739+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'715","last_scrub_stamp":"2024-09-13T05:01:06.920687+0000","last_deep_scrub":"160'715","last_deep_scrub_stamp":"2024-09-13T05:01:06.920687+0000","last_clean_scrub_stamp":"2024-09-13T05:01:06.920687+0000","objects_scrubbed":17,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:32.500994+0000","scrub_duration":0.028190621999999999,"objects_trimmed":0,"snaptrim_duration":0.0039357489999999997,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":3164,"num_read_kb":9355,"num_write":880,"num_write_kb":7841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"160'483","reported_seq":5054,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:15.232289+0000","last_change":"2024-09-13T05:01:15.232527+0000","last_active":"2024-09-13T05:01:15.232289+0000","last_peered":"2024-09-13T05:01:15.232289+0000","last_clean":"2024-09-13T05:01:15.232289+0000","last_became_active":"2024-09-13T05:00:21.960105+0000","last_became_peered":"2024-09-13T05:00:21.960105+0000","last_unstale":"2024-09-13T05:01:15.232289+0000","last_undegraded":"2024-09-13T05:01:15.232289+0000","last_fullsized":"2024-09-13T05:01:15.232289+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'483","last_scrub_stamp":"2024-09-13T05:01:15.232236+0000","last_deep_scrub":"160'483","last_deep_scrub_stamp":"2024-09-13T05:01:15.232236+0000","last_clean_scrub_stamp":"2024-09-13T05:01:15.232236+0000","objects_scrubbed":11,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:08:04.637216+0000","scrub_duration":0.021577643000000001,"objects_trimmed":0,"snaptrim_duration":0.0033058620000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5875,"num_read_kb":35757,"num_write":629,"num_write_kb":15020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":11761893,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"160'468","reported_seq":2103,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.410580+0000","last_change":"2024-09-13T05:00:57.410687+0000","last_active":"2024-09-13T05:00:57.410580+0000","last_peered":"2024-09-13T05:00:57.410580+0000","last_clean":"2024-09-13T05:00:57.410580+0000","last_became_active":"2024-09-13T04:58:03.915135+0000","last_became_peered":"2024-09-13T04:58:03.915135+0000","last_unstale":"2024-09-13T05:00:57.410580+0000","last_undegraded":"2024-09-13T05:00:57.410580+0000","last_fullsized":"2024-09-13T05:00:57.410580+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'468","last_scrub_stamp":"2024-09-13T05:00:57.410559+0000","last_deep_scrub":"160'468","last_deep_scrub_stamp":"2024-09-13T05:00:57.410559+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.410559+0000","objects_scrubbed":17,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:57:25.177746+0000","scrub_duration":0.014674352,"objects_trimmed":0,"snaptrim_duration":0.0014848890000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2995,"num_read_kb":9190,"num_write":410,"num_write_kb":5104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"160'870","reported_seq":8836,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:07.875995+0000","last_change":"2024-09-13T05:01:07.876258+0000","last_active":"2024-09-13T05:01:07.875995+0000","last_peered":"2024-09-13T05:01:07.875995+0000","last_clean":"2024-09-13T05:01:07.875995+0000","last_became_active":"2024-09-13T05:00:21.960366+0000","last_became_peered":"2024-09-13T05:00:21.960366+0000","last_unstale":"2024-09-13T05:01:07.875995+0000","last_undegraded":"2024-09-13T05:01:07.875995+0000","last_fullsized":"2024-09-13T05:01:07.875995+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'870","last_scrub_stamp":"2024-09-13T05:01:07.875938+0000","last_deep_scrub":"160'870","last_deep_scrub_stamp":"2024-09-13T05:01:07.875938+0000","last_clean_scrub_stamp":"2024-09-13T05:01:07.875938+0000","objects_scrubbed":15,"log_size":870,"log_dups_size":0,"ondisk_log_size":870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:30:45.718013+0000","scrub_duration":0.030610100000000001,"objects_trimmed":0,"snaptrim_duration":0.0014575569999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9792,"num_read_kb":27359,"num_write":1153,"num_write_kb":19490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":7351111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"160'471","reported_seq":7204,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:00:57.561850+0000","last_change":"2024-09-13T05:00:57.561976+0000","last_active":"2024-09-13T05:00:57.561850+0000","last_peered":"2024-09-13T05:00:57.561850+0000","last_clean":"2024-09-13T05:00:57.561850+0000","last_became_active":"2024-09-13T05:00:19.954785+0000","last_became_peered":"2024-09-13T05:00:19.954785+0000","last_unstale":"2024-09-13T05:00:57.561850+0000","last_undegraded":"2024-09-13T05:00:57.561850+0000","last_fullsized":"2024-09-13T05:00:57.561850+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'471","last_scrub_stamp":"2024-09-13T05:00:57.561829+0000","last_deep_scrub":"160'471","last_deep_scrub_stamp":"2024-09-13T05:00:57.561829+0000","last_clean_scrub_stamp":"2024-09-13T05:00:57.561829+0000","objects_scrubbed":22,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:02.052421+0000","scrub_duration":0.024728844,"objects_trimmed":0,"snaptrim_duration":0.0017695829999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7210,"num_read_kb":16554,"num_write":424,"num_write_kb":7402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":2726287,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"160'428","reported_seq":5212,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:16.252436+0000","last_change":"2024-09-13T05:01:16.252761+0000","last_active":"2024-09-13T05:01:16.252436+0000","last_peered":"2024-09-13T05:01:16.252436+0000","last_clean":"2024-09-13T05:01:16.252436+0000","last_became_active":"2024-09-13T04:58:03.916928+0000","last_became_peered":"2024-09-13T04:58:03.916928+0000","last_unstale":"2024-09-13T05:01:16.252436+0000","last_undegraded":"2024-09-13T05:01:16.252436+0000","last_fullsized":"2024-09-13T05:01:16.252436+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'428","last_scrub_stamp":"2024-09-13T05:01:16.252377+0000","last_deep_scrub":"160'428","last_deep_scrub_stamp":"2024-09-13T05:01:16.252377+0000","last_clean_scrub_stamp":"2024-09-13T05:01:16.252377+0000","objects_scrubbed":18,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:15:27.453430+0000","scrub_duration":0.033938712000000003,"objects_trimmed":0,"snaptrim_duration":0.001351735,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5435,"num_read_kb":31446,"num_write":402,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"160'470","reported_seq":3730,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:01.606515+0000","last_change":"2024-09-13T05:01:01.606773+0000","last_active":"2024-09-13T05:01:01.606515+0000","last_peered":"2024-09-13T05:01:01.606515+0000","last_clean":"2024-09-13T05:01:01.606515+0000","last_became_active":"2024-09-13T04:58:03.926507+0000","last_became_peered":"2024-09-13T04:58:03.926507+0000","last_unstale":"2024-09-13T05:01:01.606515+0000","last_undegraded":"2024-09-13T05:01:01.606515+0000","last_fullsized":"2024-09-13T05:01:01.606515+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'470","last_scrub_stamp":"2024-09-13T05:01:01.606449+0000","last_deep_scrub":"160'470","last_deep_scrub_stamp":"2024-09-13T05:01:01.606449+0000","last_clean_scrub_stamp":"2024-09-13T05:01:01.606449+0000","objects_scrubbed":14,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:57:08.407246+0000","scrub_duration":0.021121075999999999,"objects_trimmed":0,"snaptrim_duration":0.0021798479999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3978,"num_read_kb":13279,"num_write":437,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"160'644","reported_seq":2088,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-13T05:01:10.230566+0000","last_change":"2024-09-13T05:01:10.230869+0000","last_active":"2024-09-13T05:01:10.230566+0000","last_peered":"2024-09-13T05:01:10.230566+0000","last_clean":"2024-09-13T05:01:10.230566+0000","last_became_active":"2024-09-13T04:58:03.916106+0000","last_became_peered":"2024-09-13T04:58:03.916106+0000","last_unstale":"2024-09-13T05:01:10.230566+0000","last_undegraded":"2024-09-13T05:01:10.230566+0000","last_fullsized":"2024-09-13T05:01:10.230566+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'644","last_scrub_stamp":"2024-09-13T05:01:10.230510+0000","last_deep_scrub":"160'644","last_deep_scrub_stamp":"2024-09-13T05:01:10.230510+0000","last_clean_scrub_stamp":"2024-09-13T05:01:10.230510+0000","objects_scrubbed":14,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:02:31.739197+0000","scrub_duration":0.027995584,"objects_trimmed":0,"snaptrim_duration":0.0028391710000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3378,"num_read_kb":11036,"num_write":779,"num_write_kb":7410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":497,"num_object_clones":0,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":493,"num_read":163045,"num_read_kb":654590,"num_write":21442,"num_write_kb":341883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":73203591,"num_keys_recovered":197,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19027,"ondisk_log_size":19027,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20407,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45272,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326568,"statfs":{"total":96636764160,"available":96590405632,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":44431819},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.81200000000000006,"15min":0.81499999999999995},"min":{"1min":0.49299999999999999,"5min":0.438,"15min":0.438},"max":{"1min":2.375,"5min":2.375,"15min":2.375},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.86799999999999999,"15min":0.85399999999999998},"min":{"1min":0.48299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.88500000000000001,"15min":0.84899999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.141,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.099},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.504,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.93500000000000005,"15min":0.96199999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.92600000000000005,"15min":0.91000000000000003},"min":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.8839999999999999,"5min":3.5390000000000001,"15min":3.5390000000000001},"last":1.002}]},{"osd":3,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.94999999999999996,"15min":0.94699999999999995},"min":{"1min":0.57499999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.5409999999999999,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.98899999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.728,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.82399999999999995,"15min":0.82999999999999996},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.044,"15min":1.05},"min":{"1min":0.43099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":1.028}]},{"osd":5,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90500000000000003,"15min":0.88400000000000001},"min":{"1min":0.374,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.375,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.115,"5min":1.034,"15min":1.048},"min":{"1min":0.46999999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Sep 13 05:00:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.998,"15min":1.018},"min":{"1min":0.434,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4610000000000001,"5min":1.843,"15min":1.843},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.139,"15min":1.163},"min":{"1min":0.497,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.1279999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46000000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.2010000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.78600000000000003,"15min":0.78800000000000003},"min":{"1min":0.42199999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.755},"min":{"1min":0.38700000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.202,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.44600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.218,"5min":1.337,"15min":1.337},"last":1.1160000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.91400000000000003,"15min":0.94799999999999995},"min":{"1min":0.43099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.8700000000000001,"5min":5.343,"15min":5.343},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.48499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.85599999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.88400000000000001,"15min":0.89300000000000002},"min":{"1min":0.499,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92000000000000004,"15min":0.93799999999999994},"min":{"1min":0.38500000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.2230000000000001}]},{"osd":4,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.90600000000000003,"15min":0.94299999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.103,"5min":1.323,"15min":1.323},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95399999999999996,"15min":0.97899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.3220000000000001,"5min":1.575,"15min":1.575},"last":1.014}]},{"osd":5,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.93000000000000005,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6919999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.056,"5min":0.97499999999999998,"15min":0.97299999999999998},"min":{"1min":0.53400000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.1910000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:00:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0149999999999999,"15min":1.0289999999999999},"min":{"1min":0.435,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.605,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.077},{"interface":"front","average":{"1min":1.03,"5min":0.98599999999999999,"15min":0.97699999999999998},"min":{"1min":0.59499999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5820000000000001,"5min":1.762,"15min":1.762},"last":1.0509999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51032,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320808,"statfs":{"total":96636764160,"available":96584507392,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86499999999999999,"15min":0.90600000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.972,"5min":3.972,"15min":3.972},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.52300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.90300000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.84799999999999998},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.4730000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89300000000000002,"15min":0.91400000000000003},"min":{"1min":0.53200000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2809999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0309999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86199999999999999,"15min":0.879},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3109999999999999,"5min":1.607,"15min":1.607},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.88300000000000001,"15min":0.89500000000000002},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4350000000000001,"5min":1.579,"15min":1.579},"last":1.077}]},{"osd":3,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":1.163,"15min":1.3149999999999999},"min":{"1min":0.53300000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1919999999999999,"5min":11.266,"15min":11.266},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.0880000000000001,"5min":0.96499999999999997,"15min":0.95899999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.9009999999999998,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":0.997}]},{"osd":5,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80100000000000005,"15min":0.81100000000000005},"min":{"1min":0.49099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.256,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.878,"15min":0.89000000000000001},"min":{"1min":0.36499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":1.3580000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.86299999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88400000000000001,"15min":0.91200000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.181,"5min":1.325,"15min":1.325},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.98799999999999999,"15min":1.016},"min":{"1min":0.48099999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3640000000000001,"5min":1.752,"15min":1.752},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.97199999999999998,"15min":0.97399999999999998},"min":{"1min":0.48099999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5229999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.002,"5min":1.0149999999999999,"15min":1.0429999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.4199999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.93100000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1921024,"data_stored":911285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60399999999999998,"15min":0.628},"min":{"1min":0.30199999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.94399999999999995,"5min":1.47,"15min":1.47},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.24099999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.3149999999999999,"5min":1.552,"15min":1.552},"last":0.56399999999999995}]},{"osd":1,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65400000000000003,"15min":0.67900000000000005},"min":{"1min":0.32900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1419999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.76800000000000002,"15min":0.81000000000000005},"min":{"1min":0.34200000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1539999999999999,"5min":1.583,"15min":1.583},"last":1.0609999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.32000000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.85599999999999998,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.79800000000000004},"min":{"1min":0.38900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3979999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.85699999999999998},"min":{"1min":0.63100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.25,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.88700000000000001,"15min":0.89000000000000001},"min":{"1min":0.63,"5min":0.432,"15min":0.432},"max":{"1min":1.1080000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.58299999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3779999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96099999999999997,"15min":0.995},"min":{"1min":0.69799999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.417,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.1619999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.98399999999999999,"15min":1.002},"min":{"1min":0.64600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.335,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0289999999999999,"15min":1.0629999999999999},"min":{"1min":0.64200000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5609999999999999,"5min":1.994,"15min":1.994},"last":1.2210000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:00:56 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0109999999999999,"15min":1.044},"min":{"1min":0.66100000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.595,"5min":1.796,"15min":1.796},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.0029999999999999,"15min":1.0169999999999999},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3500000000000001,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.091}]}]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.33600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.93700000000000006,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.623},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.32000000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.2829999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.75}]},{"osd":1,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.60099999999999998,"15min":0.55700000000000005},"min":{"1min":0.34699999999999998,"5min":0.223,"15min":0.223},"max":{"1min":1.8939999999999999,"5min":3.113,"15min":3.113},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.92600000000000005,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.95399999999999996}]},{"osd":3,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.624,"15min":0.58999999999999997},"min":{"1min":0.442,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.70799999999999996,"15min":0.66900000000000004},"min":{"1min":0.43099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9199999999999999,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.90500000000000003}]},{"osd":4,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.87,"15min":0.83499999999999996},"min":{"1min":0.69999999999999996,"5min":0.374,"15min":0.374},"max":{"1min":2.0430000000000001,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":1.159},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.93000000000000005,"15min":0.94099999999999995},"min":{"1min":0.629,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3560000000000001,"5min":1.861,"15min":1.861},"last":1.268}]},{"osd":5,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.90500000000000003,"15min":0.91500000000000004},"min":{"1min":0.71499999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4119999999999999,"5min":1.627,"15min":1.627},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.05,"5min":0.94799999999999995,"15min":0.92700000000000005},"min":{"1min":0.64600000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.9430000000000001,"5min":3.327,"15min":3.327},"last":1.034}]},{"osd":6,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.99199999999999999,"15min":1.0109999999999999},"min":{"1min":0.78500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.411,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.877},{"interface":"front","average":{"1min":1.046,"5min":0.96499999999999997,"15min":0.96099999999999997},"min":{"1min":0.65000000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.405,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.0700000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.006,"15min":1.02},"min":{"1min":0.60099999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3819999999999999,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.66800000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4650000000000001,"5min":2.254,"15min":2.254},"last":1.01}]}]},{"osd":1,"up_from":15,"seq":64424509490,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59268,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312572,"statfs":{"total":96636764160,"available":96576073728,"internally_reserved":0,"allocated":1961984,"data_stored":1501653,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.58599999999999997},"min":{"1min":0.315,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.1259999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.85999999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.26900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.39000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.96399999999999997,"5min":1.407,"15min":1.407},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.208,"5min":1.401,"15min":1.401},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68300000000000005,"15min":0.67100000000000004},"min":{"1min":0.42599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1000000000000001,"5min":1.306,"15min":1.306},"last":0.97199999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.56399999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.2270000000000001,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.044},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.88800000000000001,"15min":0.91900000000000004},"min":{"1min":0.51300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0509999999999999,"5min":1.655,"15min":1.655},"last":0.91600000000000004}]},{"osd":5,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85199999999999998,"15min":0.86299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.153,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.62,"5min":0.434,"15min":0.434},"max":{"1min":1.1850000000000001,"5min":1.49,"15min":1.49},"last":1.1459999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.88900000000000001,"15min":0.90800000000000003},"min":{"1min":0.62,"5min":0.496,"15min":0.496},"max":{"1min":1.2250000000000001,"5min":1.577,"15min":1.577},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.93600000000000005,"15min":0.95299999999999996},"min":{"1min":0.65000000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3029999999999999,"5min":1.738,"15min":1.738},"last":1.0840000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:00:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.95299999999999996,"15min":0.96999999999999997},"min":{"1min":0.60499999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2290000000000001,"5min":1.766,"15min":1.766},"last":1.234},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94799999999999995,"15min":0.95499999999999996},"min":{"1min":0.66600000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.349,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.2789999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509490,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1916928,"data_stored":911266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56100000000000005,"15min":0.57999999999999996},"min":{"1min":0.26900000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.85799999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.628,"5min":1.175,"15min":1.4430000000000001},"min":{"1min":0.28399999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.1319999999999999,"5min":15.755000000000001,"15min":15.755000000000001},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":1.2390000000000001,"15min":1.492},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.1799999999999999,"5min":15.802,"15min":15.802},"last":0.92700000000000005}]},{"osd":3,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.65800000000000003,"15min":0.61799999999999999},"min":{"1min":0.35799999999999998,"5min":0.22,"15min":0.22},"max":{"1min":1.5569999999999999,"5min":2.915,"15min":2.915},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.751,"15min":0.77000000000000002},"min":{"1min":0.38400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.4019999999999999,"15min":1.663},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2350000000000001,"5min":15.859999999999999,"15min":15.859999999999999},"last":0.871},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.80300000000000005,"15min":0.77800000000000002},"min":{"1min":0.59599999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.353,"5min":3.109,"15min":3.109},"last":1.139}]},{"osd":5,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.268,"15min":1.411},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6679999999999999,"5min":11.474,"15min":11.474},"last":1.242},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.3360000000000001,"15min":1.5189999999999999},"min":{"1min":0.61799999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.611,"5min":11.528,"15min":11.528},"last":1.101}]},{"osd":6,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.93799999999999994,"15min":0.94899999999999995},"min":{"1min":0.57799999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4330000000000001,"5min":1.571,"15min":1.571},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.498,"15min":1.764},"min":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5169999999999999,"5min":15.885999999999999,"15min":15.885999999999999},"last":0.96099999999999997}]},{"osd":7,"last update":"Fri Sep 13 05:00:54 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.542,"15min":1.802},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.617,"5min":15.834,"15min":15.834},"last":1.054},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.95999999999999996,"15min":0.93100000000000005},"min":{"1min":0.61499999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.77,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":1.3260000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1978368,"data_stored":1501653,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.90900000000000003},"min":{"1min":0.48899999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2649999999999999,"5min":4.5250000000000004,"15min":4.5250000000000004},"last":1},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.82699999999999996,"15min":0.80600000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.77800000000000002,"15min":0.74099999999999999},"min":{"1min":0.46500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.75700000000000001,"15min":0.73499999999999999},"min":{"1min":0.58799999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.88900000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.48,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.94999999999999996,"15min":0.996},"min":{"1min":0.65600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.107,"5min":4.633,"15min":4.633},"last":0.745}]},{"osd":3,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.89300000000000002,"15min":0.85599999999999998},"min":{"1min":0.64300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.876,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.135,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.82799999999999996}]},{"osd":4,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89700000000000002,"15min":0.93500000000000005},"min":{"1min":0.45300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.204,"5min":4.6989999999999998,"15min":4.6989999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.88600000000000001,"15min":0.85999999999999999},"min":{"1min":0.60499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.3089999999999999,"5min":1.409,"15min":1.409},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.4159999999999999,"5min":4.0250000000000004,"15min":4.0250000000000004},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.099,"15min":1.113},"min":{"1min":0.66300000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.601,"5min":4.7599999999999998,"15min":4.7599999999999998},"last":1.04}]},{"osd":7,"last update":"Fri Sep 13 05:00:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.97499999999999998,"15min":0.94799999999999995},"min":{"1min":0.53200000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3859999999999999,"5min":1.454,"15min":1.454},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95099999999999996,"15min":0.91300000000000003},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.554,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7090,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6227,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:01:39.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T05:01:39.764 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T05:01:39.764 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T05:01:39.764 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T05:01:39.764 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:39.838 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T05:01:39.838 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T05:01:39.838 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:45.940 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T05:01:45.940 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T05:01:45.941 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:46.002 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T05:01:46.002 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T05:01:46.002 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:46.716 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-13T05:01:52.105 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T05:01:52.105 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T05:01:52.105 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.195 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T05:01:52.195 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T05:01:52.195 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.290 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T05:01:52.291 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T05:01:52.291 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.378 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T05:01:52.378 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T05:01:52.379 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.480 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T05:01:52.480 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T05:01:52.480 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T05:01:52.480 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.511 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T05:01:52.511 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T05:01:52.511 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T05:01:52.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-09-13T05:01:52.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-13T05:01:52.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-13T05:01:52.560 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T05:01:52.561 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T05:01:52.561 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T05:01:52.561 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.579 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T05:01:52.579 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T05:01:52.579 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.635 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T05:01:52.636 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T05:01:52.636 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:01:52.659 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T05:01:52.659 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T05:01:52.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-13T05:01:52.693 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T05:01:52.693 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T05:01:52.992 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T05:01:52.993 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T05:01:53.324 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T05:01:53.324 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T05:01:53.508 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T05:01:53.508 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T05:01:53.688 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T05:01:53.689 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T05:01:54.107 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T05:01:54.107 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T05:01:54.363 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T05:01:54.363 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T05:01:54.561 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T05:01:54.561 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T05:01:54.668 INFO:tasks.ceph:Archiving mon data... 2024-09-13T05:01:54.669 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/data/mon.a.tgz 2024-09-13T05:01:54.672 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-13T05:01:54.678 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.4EGV1hF4Eo 2024-09-13T05:01:54.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4EGV1hF4Eo 2024-09-13T05:01:54.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.4EGV1hF4Eo 2024-09-13T05:01:54.883 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.4EGV1hF4Eo is 398KB 2024-09-13T05:01:54.943 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.4EGV1hF4Eo 2024-09-13T05:01:54.948 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/data/mon.c.tgz 2024-09-13T05:01:54.948 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-13T05:01:54.994 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.s4DeCiH7pB 2024-09-13T05:01:54.994 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.s4DeCiH7pB 2024-09-13T05:01:55.177 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.s4DeCiH7pB 2024-09-13T05:01:55.237 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.s4DeCiH7pB is 542KB 2024-09-13T05:01:55.299 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.s4DeCiH7pB 2024-09-13T05:01:55.305 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/data/mon.b.tgz 2024-09-13T05:01:55.305 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-09-13T05:01:55.311 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.u7LZcNZDOV 2024-09-13T05:01:55.311 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.u7LZcNZDOV 2024-09-13T05:01:55.481 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.u7LZcNZDOV 2024-09-13T05:01:55.545 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.u7LZcNZDOV is 541KB 2024-09-13T05:01:55.606 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.u7LZcNZDOV 2024-09-13T05:01:55.612 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T05:01:55.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T05:01:55.614 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T05:01:55.669 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T05:01:55.669 DEBUG:teuthology.orchestra.run.smithi001:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T05:01:55.674 DEBUG:teuthology.orchestra.run.smithi144:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T05:01:55.703 INFO:teuthology.orchestra.run.smithi001.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T05:01:55.730 INFO:teuthology.orchestra.run.smithi144.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T05:01:55.734 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T05:01:55.736 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/remote/smithi001/crash 2024-09-13T05:01:55.736 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T05:01:55.759 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/remote/smithi144/crash 2024-09-13T05:01:55.760 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T05:01:55.785 INFO:tasks.ceph:Compressing logs... 2024-09-13T05:01:55.785 DEBUG:teuthology.orchestra.run.smithi001:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T05:01:55.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T05:01:57.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T05:01:57.562 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-09-13T05:01:57.562 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-13T05:02:05.748 INFO:tasks.ceph:Archiving logs... 2024-09-13T05:02:05.749 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/remote/smithi001/log 2024-09-13T05:02:05.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T05:02:06.778 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/remote/smithi144/log 2024-09-13T05:02:06.779 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T05:02:07.470 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T05:02:07.480 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T05:02:07.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T05:02:07.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T05:02:07.576 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T05:02:07.587 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T05:02:07.587 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T05:02:07.590 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T05:02:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T05:02:07.601 INFO:teuthology.orchestra.run.smithi001.stdout:============================================================================== 2024-09-13T05:02:07.601 INFO:teuthology.orchestra.run.smithi001.stdout:*hv01.front.sepi 67.205.162.81 3 u 30 64 177 0.078 -0.738 0.379 2024-09-13T05:02:07.601 INFO:teuthology.orchestra.run.smithi001.stdout:+hv02.front.sepi 63.231.80.2 3 u 35 64 177 0.072 -0.039 0.365 2024-09-13T05:02:07.601 INFO:teuthology.orchestra.run.smithi001.stdout:+hv03.front.sepi 74.6.168.72 3 u 35 64 177 0.069 -0.940 0.342 2024-09-13T05:02:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T05:02:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T05:02:07.602 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-09-13T05:02:07.602 INFO:teuthology.orchestra.run.smithi144.stdout:+hv01.front.sepi 67.205.162.81 3 u 35 64 177 0.059 -0.340 0.524 2024-09-13T05:02:07.602 INFO:teuthology.orchestra.run.smithi144.stdout:*hv02.front.sepi 63.231.80.2 3 u 36 64 177 0.093 +0.055 0.445 2024-09-13T05:02:07.602 INFO:teuthology.orchestra.run.smithi144.stdout:+hv03.front.sepi 74.6.168.72 3 u 37 64 177 0.054 -1.441 0.921 2024-09-13T05:02:07.602 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T05:02:07.603 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T05:02:07.612 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T05:02:07.613 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T05:02:07.624 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T05:02:07.635 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T05:02:07.646 INFO:teuthology.task.internal:Duration was 895.716366 seconds 2024-09-13T05:02:07.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T05:02:07.657 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T05:02:07.658 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T05:02:07.660 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T05:02:07.697 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T05:02:07.697 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-09-13T05:02:07.697 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T05:02:07.750 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-09-13T05:02:07.751 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T05:02:07.761 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T05:02:07.761 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T05:02:07.794 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T05:02:07.832 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T05:02:07.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T05:02:07.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T05:02:07.945 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T05:02:07.955 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T05:02:07.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T05:02:07.967 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T05:02:07.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T05:02:07.989 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T05:02:08.010 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T05:02:08.020 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-09-13T05:02:08.030 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-09-13T05:02:08.051 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T05:02:08.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T05:02:08.089 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T05:02:08.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T05:02:08.095 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T05:02:08.106 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T05:02:08.106 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/remote/smithi001 2024-09-13T05:02:08.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T05:02:08.159 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900825/remote/smithi144 2024-09-13T05:02:08.160 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T05:02:08.182 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T05:02:08.182 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T05:02:08.194 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T05:02:08.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T05:02:08.237 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T05:02:08.237 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T05:02:08.250 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T05:02:08.251 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T05:02:08.253 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T05:02:08.257 INFO:teuthology.orchestra.run.smithi001.stdout: 658625 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 05:02 /home/ubuntu/cephtest 2024-09-13T05:02:08.272 INFO:teuthology.orchestra.run.smithi144.stdout: 658462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 05:02 /home/ubuntu/cephtest 2024-09-13T05:02:08.274 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T05:02:08.284 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T05:02:08.310 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} duration: 895.7163655757904 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T05:02:08.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T05:02:08.393 INFO:teuthology.run:pass